Papers by rangga restu prayogo
Journal of Business and Entrepreneurship, Dec 30, 2018
ArXiv, 2021
Self-rationalization models that predict task labels and generate free-text elaborations for thei... more Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These models are, however, currently trained with a large amount of human-written free-text explanations for each task which hinders their broader usage. We propose to study a more realistic setting of self-rationalization using few training examples. We present FEB—a standardized collection of four existing Englishlanguage datasets and associated metrics. We identify the right prompting approach by extensively exploring natural language prompts on FEB. Then, by using this prompt and scaling the model size, we demonstrate that making progress on few-shot self-rationalization is possible. We show there is still ample room for improvement in this task: the average plausibility of generated explanations assessed by human annotators is at most 51%, while plausibility of human explanations is 76%. We hope that FEB togethe...
Proceedings of the 2nd International Conference of Strategic Issues on Economics, Business and, Education (ICoSIEBE 2021), 2022
Republic. community, but also beyond its boundaries and trans-generationally? Building on insight... more Republic. community, but also beyond its boundaries and trans-generationally? Building on insights from political thought, social theory, history, aesthetics, literature and visual arts, this issue provides a forum for an inclusive and reflexive debate on these questions. The papers combine theoretical reflection with in-depth analysis of case studies, ranging from ideologically and economically motivated violence in Vichy France, mass incarceration in the United States, sexual violence against women in the Former Yugoslavia and Egypt, colonial violence and violence against animals. Leebaw, too, is interested in complicity with violence, which she captures through the language of reconciliation in three different ways: 'reconciliation to one's role as a participant in, or bystander to abuse, reconciliation as self-abnegating assimilation, and reconciliation as compromise, scapegoating, or denial.' (p. XX) The first form of reconciliation/complicity is exemplified by Adolf Eichmann, the second by assimilationist refugees, and the third by soldiers who participate in war crimes. Like Mihai, Leebaw thinks of complicity temporally: 'If people feel that action is futile, they begin to make compromises that can grow larger over time.' (p. XX) Trying to understand why people 'become reconciled to what they should refuse or resist' (p. XX), she is particularly puzzled by the deadening of emotions, thoughtlessness and the failure of the imagination to inspire options beyond those that one is faced with. Building on Hannah Arendt's work, Leebaw reflects on how we can awaken emotional responsiveness and get the imagination moving to avoid problematic reconciliations to a violent and unjust reality. Against Eichmann's cog-like mentality, Leebaw argues that tragic accounts of resistance can combat resignation and despair, by inviting critical thinking and pushing our imagination to conjure alternative courses of action. Against those who internalise the role of the pariah in order to assimilate to the very communities that reject them, she proposes the antidote of pariah humour-Heine's and Chaplin's-which invites the affective identification with the marginalised 'little man'. While precariously successful, pariah humour exposes the absurdity of all human hierarchies and assimilationist aspirations that require the marginalised to renounce their identity just to be recognised as human and equal. Against the tendency to reconcile with the 'fact' of war atrocities-and the myths of heroism that obscure Western literature, Leo Tolstoy, not as a novelist, but as a Christian anarcho-pacifist. Christoyannopoulos asks a pressing question: How can our imagination be engaged critically in order to come to terms and resist our own complicity with systemic violence and oppression? The answer is through 'defamiliarisation' or ostranenie-an artistic device meant 'to shake readers into recognising the absurdity of common justifications of violence, admitting their implicit complicity in it, and noticing the art projects that tackle political violence. Garnsey theories of political and aesthetic representation to bear on ethnographic work done at the Venice Biennale, where she explored how South Africa's pavilion engaged with the issue of political violence. Garnsey examines three artworks-David Koloane's The Journey, Sue Williamson's For Thirty Years Next to His Heart and Zanele Muholi's Faces and Phases-to show that, first, at the Biennale artistic representation is enmeshed with political representation: the artists were enlisted by the state to become its cultural ambassadors. Second, the artists' representations of violence challenged the national imaginary and its entrenched myths about the past, thus undermining the representative role the state had assigned them. Koloane's The Journey chronicles Steve Biko's death at the hands of the police, and was created as the Truth and Reconciliation Commission (TRC) was hearing his murderers' application for amnesty. While their application was rejected, Biko's murderers were never prosecuted due to lack of evidence and expired statutes of limitations. Thus, the painting 'complicates the progressive narrative of the TRC by drawing attention to its limitations.' (p. XX) In contrast to Koloane's engagement with brutal violence, Williamson's For Thirty Years tackles institutional violence. It depicts a dompas-an identity document that, during Apartheid, every black South African over 16 had to carry at all times. It contained private information including the bearer's work situation, and had to be endorsed by the employer. The dompas was the instrument through which the government legally enforced spatial segregation. Williamson's piece reveals and denounces violence in its structural guise-as law that regulates and restricts access to space and work-but
Proceedings of the 2nd International Conference of Strategic Issues on Economics, Business and, Education (ICoSIEBE 2021), 2022
This study aims to analyze the feasibility of a hydroponic business on a home industry scale. Thi... more This study aims to analyze the feasibility of a hydroponic business on a home industry scale. This research tries to respond to the rising hydroponic farming system, which allows it to be applied as an alternative to a home business. However, this development is not yet optimal for new entrepreneurs because the investment value is relatively high. Meanwhile, a business feasibility study is needed to justify that the high investment value is commensurate with the profitability generated from the hydroponic business. The feasibility study results indicate that the hydroponic business with a home industry scale has excellent profitability. However, despite having a good return ratio, the income generated is relatively small as the primary income. Therefore, there is a need for continuous development in the hydroponic business to increase the production capacity of the hydroponic system. Furthermore, further studies are needed regarding market acceptance of hydroponic products to ensure the company's sustainability. Nevertheless, this study has provided empirical evidence that the hydroponic industry has potential and financial benefits as an alternative to starting a home-based business.
2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL), 2021
Author Name Disambiguation (AND) is the task of resolving which author mentions in a bibliographi... more Author Name Disambiguation (AND) is the task of resolving which author mentions in a bibliographic database refer to the same real-world person, and is a critical ingredient of digital library applications such as search and citation analysis. While many AND algorithms have been proposed, comparing them is difficult because they often employ distinct features and are evaluated on different datasets. In response to this challenge, we present S2AND, a unified benchmark dataset for AND on scholarly papers, as well as an open-source reference model implementation. Our dataset harmonizes eight disparate AND datasets into a uniform format, with a single rich feature set drawn from the Semantic Scholar (S2) database. Our evaluation suite for S2AND reports performance split by facets like publication year and number of papers, allowing researchers to track both global performance and measures of fairness across facet values. Our experiments show that because previous datasets tend to cover idiosyncratic and biased slices of the literature, algorithms trained to perform well on one on them may generalize poorly to others. By contrast, we show how training on a union of datasets in S2AND results in more robust models that perform well even on datasets unseen in training. The resulting AND model also substantially improves over the production algorithm in S2, reducing error by over 50% in terms of B 3 F1. We release our unified dataset, model code, trained models, and evaluation suite to the research community. 1 Index Terms-Digital libraries, Author name disambiguation, Out-of-domain evaluation.
Proceedings of the 25th Conference on Computational Natural Language Learning, 2021
The capabilities of today's natural language processing systems are typically evaluated using lar... more The capabilities of today's natural language processing systems are typically evaluated using large datasets of curated questions and answers. While these are critical benchmarks of progress, they also suffer from weakness due to artificial distributions and incomplete knowledge. Artifacts arising from artificial distributions can overstate language model performance, while incomplete knowledge limits fine-grained analysis. In this work, we introduce a complementary benchmarking approach based on SimPlified Language Activity Traces (SPLAT). SPLATs are corpora of language encodings of activity in some closed domain (we study traces from chess and baseball games in this work). SPLAT datasets use naturally-arising distributions, allow the generation of question-answer pairs at scale, and afford complete knowledge in their closed domains. We show that language models of three different architectures can answer questions about world states using only verb-like encodings of activity. Our approach is extensible to new language models and additional question-answering tasks.
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021
Conversations aimed at determining good recommendations are iterative in nature. People often exp... more Conversations aimed at determining good recommendations are iterative in nature. People often express their preferences in terms of a critique of the current recommendation (e.g., "It doesn't look good for a date"), requiring some degree of common sense for a preference to be inferred. In this work, we present a method for transforming a user critique into a positive preference (e.g., "I prefer more romantic") in order to retrieve reviews pertaining to potentially better recommendations (e.g., "Perfect for a romantic dinner"). We leverage a large neural language model (LM) in a fewshot setting to perform critique-to-preference transformation, and we test two methods for retrieving recommendations: one that matches embeddings, and another that fine-tunes an LM for the task. We instantiate this approach in the restaurant domain and evaluate it using a new dataset of restaurant critiques. In an ablation study, we show that utilizing critiqueto-preference transformation improves recommendations, and that there are at least three general cases that explain this improved performance.
Examining Relationship of Knowledge Sharing, Innovation Capability, Responsive Capability and Marketing Performance in Inkubator Bisnis at UPN “Veteran”East Java
Nusantara Science and Technology Proceedings, 2019
Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, 2020
Peer grading, in which students grade each other's work, can provide an educational opportunity f... more Peer grading, in which students grade each other's work, can provide an educational opportunity for students and reduce grading effort for instructors. A variety of methods have been proposed for synthesizing peer-assigned grades into accurate submission grades. However, when the assumptions behind these methods are not met, they may underperform a simple baseline of averaging the peer grades. We introduce SABTXT, which improves over previous work through two mechanisms. First, SABTXT uses a limited amount of historical instructor ground truth to model and correct for each peer's grading bias. Secondly, SABTXT models the thoroughness of a peer review based on its textual content, and puts more weight on the more thorough peer reviews when computing submission grades. In our experiments with over ten thousand peer reviews collected over four courses, we show that SABTXT outperforms existing approaches on our collected data, and achieves a mean squared error that is 6% lower than the strongest baseline on average. CCS CONCEPTS • Applied computing → Learning management systems.
ArXiv, 2020
Research in human-centered AI has shown the benefits of machine-learning systems that can explain... more Research in human-centered AI has shown the benefits of machine-learning systems that can explain their predictions. Methods that allow users to tune a model in response to the explanations are similarly useful. While both capabilities are well-developed for transparent learning models (e.g., linear models and GA2Ms), and recent techniques (e.g., LIME and SHAP) can generate explanations for opaque models, no method currently exists for tuning of opaque models in response to explanations. This paper introduces LIMEADE, a general framework for tuning an arbitrary machine learning model based on an explanation of the model's prediction. We apply our framework to Semantic Sanity, a neural recommender system for scientific papers, and report on a detailed user study, showing that our framework leads to significantly higher perceived user control, trust, and satisfaction.
ArXiv, 2021
Determining coreference of concept mentions across multiple documents is fundamental for natural ... more Determining coreference of concept mentions across multiple documents is fundamental for natural language understanding. Work on cross-document coreference resolution (CDCR) typically considers mentions of events in the news, which do not often involve abstract technical concepts that are prevalent in science and technology. These complex concepts take diverse or ambiguous forms and have many hierarchical levels of granularity (e.g., tasks and subtasks), posing challenges for CDCR. We present a new task of hierarchical CDCR for concepts in scientific papers, with the goal of jointly inferring coreference clusters and hierarchy between them. We create SCICO, an expert-annotated dataset for this task, which is 3X larger than the prominent ECB+ resource. We find that tackling both coreference and hierarchy at once outperforms disjoint models, which we hope will spur development of joint models for SCICO.1
ArXiv, 2020
Abductive reasoning is inference to the most plausible explanation. For example, if Jenny finds h... more Abductive reasoning is inference to the most plausible explanation. For example, if Jenny finds her house in a mess when she returns from work, and remembers that she left a window open, she can hypothesize that a thief broke into her house and caused the mess, as the most plausible explanation. While abduction has long been considered to be at the core of how people interpret and read between the lines in natural language (Hobbs et al., 1988), there has been relatively little research in support of abductive natural language inference and generation. We present the first study that investigates the viability of language-based abductive reasoning. We introduce a challenge dataset, ART, that consists of over 20k commonsense narrative contexts and 200k explanations. Based on this dataset, we conceptualize two new tasks -- (i) Abductive NLI: a multiple-choice question answering task for choosing the more likely explanation, and (ii) Abductive NLG: a conditional generation task for expl...
Numerous studies have demonstrated the effectiveness of pretrained contextualized language models... more Numerous studies have demonstrated the effectiveness of pretrained contextualized language models such as BERT and T5 for ad-hoc search. However, it is not well-understood why these methods are so effective, what makes some variants more effective than others, and what pitfalls they may have. We present a new comprehensive framework for Analyzing the Behavior of Neural IR ModeLs (ABNIRML), which includes new types of diagnostic tests that allow us to probe several characteristics---such as sensitivity to word order---that are not addressed by previous techniques. To demonstrate the value of the framework, we conduct an extensive empirical study that yields insights into the factors that contribute to the neural model's gains, and identify potential unintended biases the models exhibit. We find evidence that recent neural ranking models have fundamentally different characteristics from prior ranking models. For instance, these models can be highly influenced by altered document w...
Factors of tourism into one of the largest and fastest way to grow an economy is suitable for a c... more Factors of tourism into one of the largest and fastest way to grow an economy is suitable for a country that would like to expand it to include Indonesia. The development of tourism through promotion and maximum service is already undertaken by the authorities to attract domestic and foreign tourists in Indonesia. One of the attractions which are experiencing an increase in visiting Goa Pindul. The problems in Goa Pindul are increase tourists revisit intention, but the facilities and infrastructure are less well. The purpose of this research was to study the relationship between e-WOM, destination image, perceived value, and return to visit in the tourism industry. Sample techniques used in this research is purposive sampling using accidental sampling. Respondents who used as many as 200 respondents on travelers who already visited one time, aged 17 years, and using the internet to searching for information about Goa Pindul. Data analysis techniques used are PLS-SEM and Sobel test. ...
Topic models are in widespread use in natural language processing and beyond. Here, we propose a ... more Topic models are in widespread use in natural language processing and beyond. Here, we propose a new framework for the evaluation of topic modeling algorithms based on synthetic corpora containing an unambiguously defined ground truth topic structure. The major innovation of our approach is the ability to quantify the agreement between the planted and inferred topic structures by comparing the assigned topic labels at the level of the tokens. In experiments, our approach yields novel insights about the relative strengths of topic models as corpus characteristics vary, and the first evidence of an ``undetectable phase'' for topic models when the planted structure is weak. We also establish the practical relevance of the insights gained for synthetic corpora by predicting the performance of topic modeling algorithms in classification tasks in real-world corpora.
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020
Identification of new concepts in scientific literature can help power faceted search, scientific... more Identification of new concepts in scientific literature can help power faceted search, scientific trend analysis, knowledge-base construction, and more, but current methods are lacking. Manual identification can't keep up with the torrent of new publications, while the precision of existing automatic techniques is too low for many applications. We present an unsupervised concept extraction method for scientific literature that achieves much higher precision than previous work. Our approach relies on a simple but novel intuition: each scientific concept is likely to be introduced or popularized by a single paper that is disproportionately cited by subsequent papers mentioning the concept. From a corpus of computer science papers on arXiv, we find that our method achieves a Precision@1000 of 99%, compared to 86% for prior work, and a substantially better precision-yield trade-off across the top 15,000 extractions. To stimulate research in this area, we release our code and data. 1 CCS CONCEPTS • Computing methodologies → Information extraction; • Information systems → Content analysis and feature selection.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020
Language models pretrained on text from a wide variety of sources form the foundation of today's ... more Language models pretrained on text from a wide variety of sources form the foundation of today's NLP. In light of the success of these broad-coverage models, we investigate whether it is still helpful to tailor a pretrained model to the domain of a target task. We present a study across four domains (biomedical and computer science publications, news, and reviews) and eight classification tasks, showing that a second phase of pretraining indomain (domain-adaptive pretraining) leads to performance gains, under both high-and low-resource settings. Moreover, adapting to the task's unlabeled data (task-adaptive pretraining) improves performance even after domain-adaptive pretraining. Finally, we show that adapting to a task corpus augmented using simple data selection strategies is an effective alternative, especially when resources for domain-adaptive pretraining might be unavailable. Overall, we consistently find that multiphase adaptive pretraining offers large gains in task performance.
Findings of the Association for Computational Linguistics: EMNLP 2020, 2020
Recent advances in commonsense reasoning depend on large-scale human-annotated training sets to a... more Recent advances in commonsense reasoning depend on large-scale human-annotated training sets to achieve peak performance. However, manual curation of training sets is expensive and has been shown to introduce annotation artifacts that neural models can readily exploit and overfit to. We propose a novel generative data augmentation technique, G-DAUG c , that aims to achieve more accurate and robust learning in a low-resource setting. Our approach generates synthetic examples using pretrained language models, and selects the most informative and diverse set of examples for data augmentation. On experiments with multiple commonsense reasoning benchmarks, G-DAUG c consistently outperforms existing data augmentation methods based on back-translation, establishing a new state-of-the-art on WINOGRANDE, CODAH, and COMMONSENSEQA, and also enhances out-of-distribution generalization, proving to be more robust against adversaries or perturbations. Our analysis demonstrates that G-DAUG c produces a diverse set of fluent training examples, and that its selection and training approaches are important for performance.
Proceedings of the AAAI Conference on Artificial Intelligence, 2020
Neural network language models (NNLMs) have achieved ever-improving accuracy due to more sophisti... more Neural network language models (NNLMs) have achieved ever-improving accuracy due to more sophisticated architectures and increasing amounts of training data. However, the inductive bias of these models (formed by the distributional hypothesis of language), while ideally suited to modeling most running text, results in key limitations for today's models. In particular, the models often struggle to learn certain spatial, temporal, or quantitative relationships, which are commonplace in text and are second-nature for human readers. Yet, in many cases, these relationships can be encoded with simple mathematical or logical expressions. How can we augment today's neural models with such encodings?In this paper, we propose a general methodology to enhance the inductive bias of NNLMs by incorporating simple functions into a neural architecture to form a hierarchical neural-symbolic language model (NSLM). These functions explicitly encode symbolic deterministic relationships to form ...
Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*, 2019
We introduce a novel topic modeling approach based on constructing a semantic set cover for clust... more We introduce a novel topic modeling approach based on constructing a semantic set cover for clusters of similar documents. Specifically, our approach first clusters documents using their Tf-Idf representation, and then covers each cluster with a set of topic words based on semantic similarity, defined in terms of a word embedding. Computing a topic cover amounts to solving a minimum set cover problem. Our evaluation compares our topic modeling approach to Latent Dirichlet Allocation (LDA) on three metrics: 1) qualitative topic match, measured using evaluations by Amazon Mechanical Turk (MTurk) workers, 2) performance on classification tasks using each topic model as a sparse feature representation, and 3) topic coherence. We find that qualitative judgments significantly favor our approach, the method outperforms LDA on topic coherence, and is comparable to LDA on document classification tasks.
Uploads
Papers by rangga restu prayogo