Anupam Jamatia
580 California St., Suite 400
San Francisco, CA, 94104
Validation of Facts Against Textual Sources
2019, Proceedings - Natural Language Processing in a Deep Learning World
https://doi.org/10.26615/978-954-452-056-4_104
Sign up for access to the world's latest research
Abstract
In today's world, the spreading of fake news has become facile through social media which diffuses rapidly and can be believed easily. Fact Checkers or Fact Verifiers are the need of the hour. In this paper, we propose a system which would verify a claim(fact) against a textual source provided and classify the claim to be true, false, out-of-context or inappropriate with respect to that source. This would help us to verify a fact as well as know about the source of our knowledge base against which the fact is being verified. We used a two-step approach to achieve our goal. First step is about retrieving the evidence related to the claims from the textual source. Next step is the classification of the claim as true, false, inappropriate and out of context with respect to the evidence using a modified version of textual entailment module. The accuracy of the best performing system is 64.95%.
References (23)
- Tariq Alhindi, Savvas Petridis, and Smaranda Mure- san. 2018. Where is your evidence: Improving fact- checking by justification modeling. In Proceedings of the First Workshop on Fact Extraction and VERi- fication (FEVER). pages 85-90.
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Associa- tion for Computational Linguistics, Lisbon, Portu- gal, pages 632-642.
- Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017a. Reading wikipedia to answer open- domain questions. In Proceedings of the 55th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, pages 1870-1879.
- Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017b. Enhanced LSTM for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers). As- sociation for Computational Linguistics, Vancouver, Canada, pages 1657-1668.
- William Ferreira and Andreas Vlachos. 2016. Emer- gent: a novel data-set for stance classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technolo- gies. Association for Computational Linguistics, San Diego, California, pages 1163-1168.
- J. L. Fleiss. 1971. Measuring nominal scale agree- ment among many raters. Psychological Bulletin 76(5):378-382.
- Yichen Gong, Heng Luo, and Jian Zhang. 2018. Nat- ural language inference over interaction space. In International Conference on Learning Representa- tions.
- Christopher Hidey and Mona Diab. 2018. Team SWEEPer: Joint sentence extraction and fact check- ing with pointer networks. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER). Brussels, Belgium, pages 150-155.
- Djoerd Hiemstra. 2000. A probabilistic justification for using tfxidf term weighting in information retrieval. International Journal on Digital Libraries 3(2):131- 139.
- Nayeon Lee, Chien-Sheng Wu, and Pascale Fung. 2018. Improving large-scale fact-checking using decomposable attention models and lexical tagging. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing. pages 1133-1138.
- Marco Marelli, Luisa Bentivogli, Marco Baroni, Raf- faella Bernardi, Stefano Menini, and Roberto Zam- parelli. 2014. Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Association for Computational Linguistics, Dublin, Ireland.
- Yixin Nie and Mohit Bansal. 2017. Shortcut-stacked sentence encoders for multi-domain inference. In Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP. Association for Computational Linguistics, Copenhagen, Den- mark, pages 41-45.
- Yixin Nie, Haonan Chen, and Mohit Bansal. 2018. Combining fact extraction and verification with neu- ral semantic matching networks. arXiv preprint arXiv:1811.07039 .
- Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceed- ings of the 2016 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2249-2255.
- Aniketh Janardhan Reddy, Gil Rocha, and Diego Es- teves. 2018. Defactonlp: Fact verification using en- tity recognition, tfidf vector comparison and decom- posable attention. In Proceedings of the First Work- shop on Fact Extraction and VERification (FEVER). Association for Computational Linguistics, pages 132-137.
- Benjamin Riedel, Isabelle Augenstein, Georgios P Sp- ithourakis, and Sebastian Riedel. 2017. A sim- ple but tough-to-beat baseline for the fake news challenge stance detection task. arXiv preprint arXiv:1707.03264 .
- Mark Sammons, Vinod Vydiswaran, and Dan Roth. 2012. Recognizing textual entailment. Multilin- gual Natural Language Applications: From Theory to Practice pages 209-258.
- James Thorne and Andreas Vlachos. 2018. Automated fact checking: Task formulations, methods and fu- ture directions. In Proceedings of the 27th Inter- national Conference on Computational Linguistics. Association for Computational Linguistics, pages 3346-3359.
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018a. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, New Orleans, Louisiana, pages 809-819.
- James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2018b. The Fact Extraction and VERification (FEVER) shared task. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER).
- James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2018c. Proceedings of the first workshop on fact extraction and verification (fever). In Proceedings of the First Workshop on Fact Extraction and VERi- fication (FEVER).
- Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Lan- guage Technologies and Computational Social Sci- ence. pages 18-22.
- William Yang Wang. 2017. "liar, liar pants on fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers). Association for Computational Lin- guistics, Vancouver, Canada, pages 422-426.