Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Data
Other Analysis
Related Work
Conclusion
References
All Topics
Computer Science

Towards Automatic Generation of Questions from Long Answers

Profile image of shlok mishrashlok mishra

2020

Abstract

Automatic question generation (AQG) has broad applicability in domains such as tutoring systems, conversational agents, healthcare literacy, and information retrieval. Existing efforts at AQG have been limited to short answer lengths of up to two or three sentences. However, several real-world applications require question generation from answers that span several sentences. Therefore, we propose a novel evaluation benchmark to assess the performance of existing AQG systems for long-text answers. We leverage the large-scale open-source Google Natural Questions dataset to create the aforementioned long-answer AQG benchmark. We empirically demonstrate that the performance of existing AQG methods significantly degrades as the length of the answer increases. Transformer-based methods outperform other existing AQG methods on long answers in terms of automatic as well as human evaluation. However, we still observe degradation in the performance of our best performing models with increasin...

Related papers

ParaQG: A System for Generating Questions and Answers from Paragraphs
Vishwajeet Kumar

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, 2019

Generating syntactically and semantically valid and relevant questions from paragraphs is useful with many applications. Manual generation is a labour-intensive task, as it requires the reading, parsing and understanding of long passages of text. A number of question generation models based on sequence-to-sequence techniques have recently been proposed. Most of them generate questions from sentences only, and none of them is publicly available as an easy-to-use service. In this paper, we demonstrate ParaQG, a Web-based system for generating questions from sentences and paragraphs. ParaQG incorporates a number of novel functionalities to make the question generation process user-friendly. It provides an interactive interface for a user to select answers with visual insights on generation of questions. It also employs various faceted views to group similar questions as well as filtering techniques to eliminate unanswerable questions.

View PDFchevron_right
Answer-driven Deep Question Generation based on Reinforcement Learning
liuyin wang

Proceedings of the 28th International Conference on Computational Linguistics

Deep question generation (DQG) aims to generate complex questions through reasoning over multiple documents. The task is challenging and underexplored. Existing methods mainly focus on enhancing document representations, with little attention paid to the answer information, which may result in the generated question not matching the answer type and being answerirrelevant. In this paper, we propose an Answer-driven Deep Question Generation (ADDQG) model based on the encoder-decoder framework. The model makes better use of the target answer as a guidance to facilitate question generation. First, we propose an answer-aware initialization module with a gated connection layer which introduces both document and answer information to the decoder, thus helping to guide the choice of answer-focused question words. Then a semantic-rich fusion attention mechanism is designed to support the decoding process, which integrates the answer with the document representations to promote the proper handling of answer information during generation. Moreover, reinforcement learning is applied to integrate both syntactic and semantic metrics as the reward to enhance the training of the ADDQG. Extensive experiments on the HotpotQA dataset show that ADDQG outperforms state-of-the-art models in both automatic and human evaluations.

View PDFchevron_right
Question generation shared task and evaluation challenge: status report
Cristian Petre Moldovan

2011

The First Shared Task Evaluation Challenge on Question Generation took place in 2010 as part of the 3 rd workshop on Question Generation. The campaign included two tasks: Question Generation from Sentences and Question Generation from Paragraphs. This status report briefly summarizes the motivation, tasks and results. Lessons learned relevant to future QG-STECs are also offered.

View PDFchevron_right
Improving Question Answering with Generation of NQ-like Questions
Abhranil Chandra

Cornell University - arXiv, 2022

Question Answering (QA) systems require a large amount of annotated data which is costly and time-consuming to gather. Converting datasets of existing QA benchmarks are challenging due to different formats and complexities. To address these issues, we propose an algorithm to automatically generate shorter questions resembling day-today human communication in the Natural Questions (NQ) dataset from longer trivia questions in Quizbowl (QB) dataset by leveraging conversion in style among the datasets. This provides an automated way to generate more data for our QA systems. To ensure quality as well as quantity of data, we detect and remove illformed questions using a neural classifier. We demonstrate that in a low resource setting, using the generated data improves the QA performance over the baseline system on both NQ and QB data. Our algorithm improves the scalability of training data while maintaining quality of data for QA systems.

View PDFchevron_right
Controllable Question Generation via Sequence-to-Sequence Neural Model with Auxiliary Information
Andy Khong

2020 International Joint Conference on Neural Networks (IJCNN)

Automatic question generation (QG) has found applications in the education sector and to enhance human-machine interactions in chatbots. Existing neural QG models can be categorized into answer-unaware and answer-aware models. One of the main challenges faced by existing neural QG models is the degradation in performance due to the issue of one-tomany mapping, where, given a passage, both answer (query interest/question intent) and auxiliary information (context information present in the question) can result in different questions being generated. We propose a controllable question generation model (CQG) that employs an attentive sequence-to-sequence (seq2seq) based generative model with copying mechanism. The proposed CQG also incorporates query interest and auxiliary information as controllers to address the one-to-many mapping problem in QG. Two variants of embedding strategies are designed for CQG to achieve good performance. To verify its performance, an automatic labeling scheme for harvesting auxiliary information is first developed. A QG dataset is also annotated with auxiliary information from a reading comprehension dataset. Performance evaluation shows that the proposed model not only outperforms existing QG models, it also has the potential to generate multiple questions that are relevant given a single passage.

View PDFchevron_right
Quiz-Style Question Generation for News Stories
Vinh Tran

Proceedings of the Web Conference 2021, 2021

A large majority of American adults get at least some of their news from the Internet. Even though many online news products have the goal of informing their users about the news, they lack scalable and reliable tools for measuring how well they are achieving this goal, and therefore have to resort to noisy proxy metrics (e.g., clickthrough rates or reading time) to track their performance. As a first step towards measuring news informedness at a scale, we study the problem of quiz-style multiple-choice question generation, which may be used to survey users about their knowledge of recent news. In particular, we formulate the problem as two sequence-to-sequence tasks: question-answer generation (QAG) and distractor, or incorrect answer, generation (DG). We introduce NewsQuizQA, the first dataset intended for quiz-style questionanswer generation, containing 20K human written question-answer pairs from 5K news article summaries. Using this dataset, we propose a series of novel techniques for applying large pre-trained Transformer encoder-decoder models, namely PEGASUS and T5, to the tasks of question-answer generation and distractor generation. We show that our models outperform strong baselines using both automated metrics and human raters. We provide a case study of running weekly quizzes on real-world users via the Google Surveys platform over the course of two months. We found that users generally found the automatically generated questions to be educational and enjoyable. Finally, to serve the research community, we are releasing the NewsQuizQA dataset. CCS CONCEPTS • Computing methodologies → Natural language generation.

View PDFchevron_right
Encoder-Decoder Architectures for Generating Questions
Yashvardhan Sharma

Procedia Computer Science, 2018

With exploding textual data on the internet with e-books, legal documents and products information, it is an opportunity to harness it for applications which can aid human tasks. Developing systems for question generation can be used for making frequently-askedquestions, creating school quiz-es and serve for the purpose of unified AI. Here in this study various encoder decoder architectures for generating questions from text inputs have been explored using Stanford's SQuAD dataset as for training development and test sets and evaluation metrics such as BLEU, ROUGUE and training time were used to compare the effectiveness of the models. The article develops upon the work of current end-to-end system by using gated recurrent unit in place of long short term memory which give similar accuracy but with lesser training time, further it also show the successfully use of a convolution based encoder for this task which gives results comparable to current state of the art system with much lesser training time.

View PDFchevron_right
Question-Worthy Sentence Selection for Question Generation
Aijun An

Advances in Artificial Intelligence, 2020

The problem of automatic question generation from text is of increasing importance due to many useful applications. While deep neural networks achieved success in generating questions from text paragraphs, they mainly focused on a whole paragraph in generating questions, assuming all sentences are question-worthy sentences. However, a text paragraph often contains only a few important sentences that are worthy of asking questions. To that end, we present a feature-based sentence selection method for identifying question-worthy sentences. Such sentences are then used by a sequence-to-sequence (i.e., seq2seq) model to generate questions. Our experiments show that these features significantly improves the question generated by seq2seq models.

View PDFchevron_right
On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation
Jishnu Ray Chowdhury

ArXiv, 2022

We study the task of predicting a set of salient questions from a given paragraph without any prior knowledge of the precise answer. We make two main contributions. First, we propose a new method to evaluate a set of predicted questions against the set of references by using the Hungarian algorithm to assign predicted questions to references before scoring the assigned pairs. We show that our proposed evaluation strategy has better theoretical and practical properties compared to prior methods because it can properly account for the coverage of references. Second, we compare different strategies to utilize a pre-trained seq2seq model to generate and select a set of questions related to a given paragraph. The code is available1.

View PDFchevron_right
Scalable Educational Question Generation with Pre-trained Language Models
Sahan Bulla Bulathwela

arXiv (Cornell University), 2023

The automatic generation of educational questions will play a key role in scaling online education, enabling self-assessment at scale when a global population is manoeuvring their personalised learning journeys. We develop EduQG, a novel educational question generation model built by adapting a large language model. Our extensive experiments demonstrate that EduQG can produce superior educational questions by further pre-training and fine-tuning a pre-trained language model on the scientific text and science question data.

View PDFchevron_right

References (63)

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proc. ICLR.
  2. Nirmal Baral, Bishnu Hari Paudel, BK Das, Mad- hukar Aryal, Balbhadra Prasad Das, Nilambar Jha, and Madhab Lamsal. 2007. An evaluation of training of teachers in medical education in four medical schools of nepal. Nepal Med Coll J, 9(3):157-61.
  3. Jonathan C Brown, Gwen A Frishkoff, and Max- ine Eskenazi. 2005. Automatic question genera- tion for vocabulary assessment. In Proceedings of the conference on Human Language Technol- ogy and Empirical Methods in Natural Language Processing, pages 819-826. Association for Com- putational Linguistics.
  4. Yllias Chali and Sadid A Hasan. 2015. Towards topic-to-question generation. Computational Linguistics, 41(1):1-20.
  5. Guanliang Chen, Jie Yang, Claudia Hauff, and Geert-Jan Houben. 2018. Learningq: a large- scale dataset for educational question genera- tion. In Twelfth International AAAI Conference on Web and Social Media.
  6. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine transla- tion. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 1724-1734.
  7. Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860.
  8. Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. 2018. Uni- versal transformers. CoRR, abs/1807.03819.
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language un- derstanding. arXiv preprint arXiv:1810.04805.
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
  11. Xinya Du and Claire Cardie. 2017. Identifying where to focus in reading comprehension for neu- ral question generation. In Proceedings of the 2017 Conference on Empirical Methods in Nat- ural Language Processing, pages 2067-2073.
  12. Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), volume 1, pages 1342-1352.
  13. Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Güney, Volkan Cirik, and Kyunghyun Cho. 2017. Searchqa: A new q&a dataset augmented with context from a search engine. CoRR, abs/1704.05179.
  14. Sam Ganzfried and Farzana Yusuf. 2018. Optimal weighting for exam composition. Education Sci- ences, 8(1):36.
  15. Yifan Gao, Jianan Wang, Lidong Bing, Irwin King, and Michael R Lyu. 2018. Difficulty controllable question generation for reading comprehension. arXiv preprint arXiv:1807.03586.
  16. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1631-1640.
  17. Wei He, Kai Liu, Jing Liu, Yajuan Lyu, Shiqi Zhao, Xinyan Xiao, Yuan Liu, Yizhong Wang, Hua Wu, Qiaoqiao She, et al. 2018. Dureader: a chinese machine reading comprehension dataset from real-world applications. In Proceedings of the Workshop on Machine Reading for Question Answering, pages 37-46.
  18. Michael Heilman. 2011. Automatic factual ques- tion generation from text. Language Tech- nologies Institute School of Computer Science Carnegie Mellon University, 195.
  19. Michael Heilman and Noah A. Smith. 2010. Good question! statistical ranking for question gener- ation. In HLT-NAACL.
  20. Thomas Holme. 2003. Assessment and quality con- trol in chemistry education. Journal of Chemi- cal Education, 80(6):594.
  21. Wenpeng Hu, Bing Liu, Jinwen Ma, Dongyan Zhao, and Rui Yan. 2018. Aspect-based question gen- eration.
  22. A Jagannatha. 2016. H yu. bidirectional rnn for medical event detection in electronic health records. In Proceedings of the Association for Computational Linguistics Conference, pages 12-17.
  23. Yanghoon Kim, Hwanhee Lee, Joongbo Shin, and Kyomin Jung. 2019. Improving neural question generation using answer separation. In Proceed- ings of the AAAI Conference on Artificial Intel- ligence, volume 33, pages 6602-6609.
  24. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR, abs/1412.6980.
  25. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proc. ACL.
  26. Tomas Kocisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, and Edward Grefenstette. 2018. The narra- tiveqa reading comprehension challenge. Trans- actions of the Association of Computational Lin- guistics, 6:317-328.
  27. Pascal Kuyten, Timothy Bickmore, Svetlana Stoy- anchev, Paul Piwek, Helmut Prendinger, and Mitsuru Ishizuka. 2012. Fully automated gen- eration of question-answer pairs for scripted vir- tual instruction. In International Conference on Intelligent Virtual Agents, pages 1-14. Springer.
  28. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Rhinehart, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, et al. 2019. Nat- ural questions: a benchmark for question an- swering research.
  29. Igor Labutov, Sumit Basu, and Lucy Vanderwende. 2015. Deep questions without deep understand- ing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 889-898.
  30. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. Race: Large- scale reading comprehension dataset from exam- inations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Pro- cessing, pages 785-794.
  31. John P Lalor, Hao Wu, Li Chen, Kathleen M Ma- zor, and Hong Yu. 2018. Comprehenotes, an instrument to assess patient reading comprehen- sion of electronic health record notes: develop- ment and validation. Journal of medical Inter- net research, 20(4):e139.
  32. Nguyen-Thinh Le and Niels Pinkwart. 2015. Eval- uation of a question generation approach using semantic web for supporting argumentation. Re- search and practice in technology enhanced learn- ing, 10(1):3.
  33. Jindřich Libovickỳ, Jindřich Helcl, and David Mareček. 2018. Input combination strategies for multi-source transformer decoder. In Proceed- ings of the Third Conference on Machine Trans- lation: Research Papers, pages 253-260.
  34. Samuel A Livingston. 2009. Constructed-response test questions: Why we use them; how we score them. r&d connections. number 11. Educational Testing Service.
  35. Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412-1421.
  36. Karen Mazidi and Rodney D Nielsen. 2014a. Lin- guistic considerations in automatic question gen- eration. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 321-326.
  37. Karen Mazidi and Rodney D. Nielsen. 2014b. Lin- guistic considerations in automatic question gen- eration. In ACL.
  38. Shashi Narayan, Shay B Cohen, and Mirella La- pata. 2018. Ranking sentences for extractive summarization with reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technolo- gies, Volume 1 (Long Papers), volume 1, pages 1747-1759.
  39. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. choice, 2640:660.
  40. John C Ory. 1983. Improving your test questions.
  41. Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018. Scaling neural machine translation. ArXiv, abs/1806.00187.
  42. Anusri Pampari, Preethi Raghavan, Jennifer Liang, and Jian Peng. 2018. emrqa: A large cor- pus for question answering on electronic medical records. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Pro- cessing, pages 2357-2368.
  43. Liangming Pan, Wenqiang Lei, Tat-Seng Chua, and Min-Yen Kan. 2019. Recent advances in neural question generation. arXiv preprint arXiv:1905.08949.
  44. Alec Radford. 2018. Improving language under- standing by generative pre-training.
  45. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Lan- guage models are unsupervised multitask learn- ers.
  46. Preethi Raghavan, Siddharth Patwardhan, Jen- nifer J Liang, and Murthy V Devarakonda. 2018. Annotating electronic medical records for question answering. arXiv preprint arXiv:1805.06816.
  47. Pranav Rajpurkar, Jian Zhang, Konstantin Lopy- rev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-2392.
  48. Andrés Rosso-Mateus, Fabio A González, and Manuel Montes-y Gómez. 2017. A two-step neu- ral network approach to passage retrieval for open domain question answering. In Iberoameri- can Congress on Pattern Recognition, pages 566- 574. Springer.
  49. Vasile Rus, Brendan Wyse, Paul Piwek, Mihai Lintean, Svetlana Stoyanchev, and Christian Moldovan. 2010. The first question generation shared task evaluation challenge. In Proceedings of the 6th International Natural Language Gen- eration Conference.
  50. Mrinmaya Sachan and Eric Xing. 2018. Self- training for jointly learning to ask and answer questions. In Proceedings of the 2018 Confer- ence of the North American Chapter of the As- sociation for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long Papers), volume 1, pages 629-640.
  51. Patricia Shapley. 2000. On-line education to de- velop complex reasoning skills in organic chem- istry. Journal of Asynchronous Learning Net- works, 4(2):43-52.
  52. Linfeng Song, Zhiguo Wang, Wael Hamza, Yue Zhang, and Daniel Gildea. 2018. Leveraging context information for natural question gener- ation. In Proceedings of the 2018 Conference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Language Technologies, Volume 2 (Short Pa- pers), volume 2, pages 569-574.
  53. Amanda Stent, Matthew Marge, and Mohit Sing- hai. 2005. Evaluating evaluation methods for generation in the presence of variation. In Inter- national Conference on Intelligent Text Process- ing and Computational Linguistics, pages 341- 351. Springer.
  54. Sandeep Subramanian, Tong Wang, Xingdi Yuan, Saizheng Zhang, Adam Trischler, and Yoshua Bengio. 2018. Neural models for key phrase ex- traction and question generation. In Proceedings of the Workshop on Machine Reading for Ques- tion Answering, pages 78-88.
  55. Xingwu Sun, Jing Liu, Yajuan Lyu, Wei He, Yan- jun Ma, and Shi Wang. 2018. Answer-focused and position-aware neural question generation. In EMNLP.
  56. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. Newsqa: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 191-200.
  57. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. At- tention is all you need. In Advances in Neural In- formation Processing Systems, pages 5998-6008.
  58. Zichao Wang, Andrew S Lan, Weili Nie, Andrew E Waters, Phillip J Grimaldi, and Richard G Bara- niuk. 2018. Qg-net: a data-driven question gen- eration model for educational content. In Pro- ceedings of the Fifth Annual ACM Conference on Learning at Scale, page 7. ACM.
  59. Yi Yang, Wen tau Yih, and Christopher Meek. 2015. Wikiqa: A challenge dataset for open- domain question answering. In EMNLP.
  60. Xingdi Yuan, Tong Wang, Caglar Gulcehre, Alessandro Sordoni, Philip Bachman, Saizheng Zhang, Sandeep Subramanian, and Adam Trischler. 2017. Machine comprehension by text- to-text neural question generation. In Proceed- ings of the 2nd Workshop on Representation Learning for NLP, pages 15-25.
  61. Rachel Yudkowsky, Yoon Soo Park, and Steven M Downing. 2019. Assessment in health profes- sions education. Routledge.
  62. Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, and Qifa Ke. 2018. Paragraph-level neural ques- tion generation with maxout pointer and gated self-attention networks. In Proceedings of the 2018 Conference on Empirical Methods in Nat- ural Language Processing, pages 3901-3910.
  63. Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural ques- tion generation from text: A preliminary study. In National CCF Conference on Natural Lan- guage Processing and Chinese Computing, pages 662-671. Springer.

Related papers

Transformer-based End-to-End Question Generation
Charibeth Cheng

ArXiv, 2020

Question Generation (QG) is an important task in Natural Language Processing (NLP) that involves generating questions automatically when given a context paragraph. While many techniques exist for the task of QG, they employ complex model architectures, extensive features, and additional mechanisms to boost model performance. In this work, we show that transformer-based finetuning techniques can be used to create robust question generation systems using only a single pretrained language model, without the use of additional mechanisms, answer metadata, and extensive features. Our best model outperforms previous more complex RNN-based Seq2Seq models, with an 8.62 and a 14.27 increase in METEOR and ROUGE_L scores, respectively. We show that it also performs on par with Seq2Seq models that employ answer-awareness and other special mechanisms, despite being only a single-model system. We analyze how various factors affect the model's performance, such as input data formatting, the len...

View PDFchevron_right
Exploring Answer Information Methods for Question Generation with Transformers
GRISHMA SHARMA

arXiv (Cornell University), 2023

There has been a lot of work in question generation where different methods to provide target answers as input, have been employed. This experimentation has been mostly carried out for RNN based models. We use three different methods and their combinations for incorporating answer information and explore their effect on several automatic evaluation metrics. The methods that are used are answer prompting, using a custom product method using answer embeddings and encoder outputs, choosing sentences from the input paragraph that have answer related information, and using a separate cross-attention attention block in the decoder which attends to the answer. We observe that answer prompting without any additional modes obtains the best scores across rouge, meteor scores. Additionally, we use a custom metric to calculate how many of the generated questions have the same answer, as the answer which is used to generate them.

View PDFchevron_right
Simplifying Paragraph-level Question Generation via Transformer Language Models
Diane Cruz

2021

Question generation (QG) is a natural language generation task where a model is trained to ask questions corresponding to some input text. Most recent approaches frame QG as a sequence-to-sequence problem and rely on additional features and mechanisms to increase performance; however, these often increase model complexity, and can rely on auxiliary data unavailable in practical use. A single Transformer-based unidirectional language model leveraging transfer learning can be used to produce high quality questions while disposing of additional task-specific complexity. Our QG model, finetuned from GPT-2 Small, outperforms several paragraph-level QG baselines on the SQuAD dataset by 0.95 METEOR points. Human evaluators rated questions as easy to answer, relevant to their context paragraph, and corresponding well to natural human speech. Also introduced is a new set of baseline scores on the RACE dataset, which has not previously been used for QG tasks. Further experimentation with vary...

View PDFchevron_right
Vocabulary Matters: A Simple yet Effective Approach to Paragraph-level Question Generation
Vishwajeet Kumar

2020

Question generation (QG) has recently attracted considerable attention. Most of the current neural models take as input only one or two sentences, and perform poorly when multiple sentences or complete paragraphs are given as input. However, in real-world scenarios it is very important to be able to generate high-quality questions from complete paragraphs. In this paper, we present a simple yet effective technique for answer-aware question generation from paragraphs. We augment a basic sequence-to-sequence QG model with dynamic, paragraph-specific dictionary and copy attention that is persistent across the corpus, without requiring features generated by sophisticated NLP pipelines or handcrafted rules. Our evaluation on SQuAD shows that our model significantly outperforms current state-of-the-art systems in question generation from paragraphs in both automatic and human evaluation. We achieve a 6-point improvement over the best system on BLEU-4, from 16.38 to 22.62.

View PDFchevron_right
Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus
Haolan Chen

Proceedings of The Web Conference 2020, 2020

The ability to ask questions is important in both human and machine intelligence. Learning to ask questions helps knowledge acquisition, improves question-answering and machine reading comprehension tasks, and helps a chatbot to keep the conversation flowing with a human. Existing question generation models are ineffective at generating a large amount of high-quality question-answer pairs from unstructured text, since given an answer and an input passage, question generation is inherently a one-to-many mapping. In this paper, we propose Answer-Clue-Style-aware Question Generation (ACS-QG), which aims at automatically generating high-quality and diverse question-answer pairs from unlabeled text corpus at scale by imitating the way a human asks questions. Our system consists of: i) an information extractor, which samples from the text multiple types of assistive information to guide question generation; ii) neural question generators, which generate diverse and controllable questions, leveraging the extracted assistive information; and iii) a neural quality controller, which removes low-quality generated data based on text entailment. We compare our question generation models with existing approaches and resort to voluntary human evaluation to assess the quality of the generated question-answer pairs. The evaluation results suggest that our system dramatically outperforms state-of-the-art neural question generation models in terms of the generation quality, while being scalable in the meantime. With models trained on a relatively smaller amount of data, we can generate 2.8 million quality-assured question-answer pairs from a million sentences found in Wikipedia. CCS CONCEPTS • Computing methodologies → Natural language processing; Natural language generation; Machine translation.

View PDFchevron_right

Related topics

  • Computer Science
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved