The Turing Deception

Chats Between Bots: A Real-World Experiment in Writing, Recursion, and Reflexivity, 2024

The abstract and keywords below were produced by a publicly available chatbot (setting aside this first, italicized portion). The paper was inspired by a series of unsolicited emails to a professor from prospective PhD students. Suspecting some of the emails were chatbot-generated, he decided to undertake further investigation. This paper presents a reflexive exploration of large language models (LLMs) by engaging a chatbot in a recursive dialogue on AI's social, ethical, and political dimensions. The study probes how LLMs, such as chatbots, mediate academic and public discourse, emphasizing the potential shifts in knowledge production, authorship, and authority. By experimenting with chatbot-generated text, the author assesses the recursive structures and potential biases that arise when AI participates in human communication systems. The findings reflect on the ambiguous boundaries of authorship and intellectual integrity in an age of AIassisted writing, raising questions about societal implications and the efficacy of governing such technology. This work underscores the complex dynamics of AI-mediated communication, proposing that more nuanced forms of oversight and ethical consideration are essential as LLMs continue to scale within knowledge systems.

ACM Transactions on Asian and Low-Resource Language Information Processing ‎, 2024

The surge in advancements in large language models (LLMs) has expedited the generation of synthetic text imitating human writing styles. This, however, raises concerns about the potential misuse of synthetic textual data, which could compromise trust in online content. Against this backdrop, the present research aims to address the key challenges of detecting LLMs-generated texts. In this study, we used ChatGPT (v 3.5) because of its widespread and capability to comprehend and keep conversational context, allowing it to produce meaningful and contextually suitable responses. The problem revolves around the task of discerning between authentic and artificially generated textual content. To tackle this problem, we first created a dataset containing both real and DeepFake text. Subsequently, we employed transfer-learning (TL) and conducted DeepFakedetection utilizing SOTA large pre-trained LLMs. Furthermore, we conducted validation using benchmark datasets comprising unseen data samples to ensure that the model's performance reflects its ability to generalize to new data. Finally, we discussed this study's theoretical contributions, practical implications, limitations and potential avenues for future research, aiming to formulate strategies for identifying and detecting large-generative-models' produced texts. The results were promising, with accuracy ranging from 94% to 99%. The comparison between automatic detection and the human ability to detect DeepFake text revealed a significant gap in the human capacity for its identification, emphasizing an increasing need for sophisticated automated detectors. The investigation into AI-generated content detection holds central importance in the age of LLMs and technology convergence. This study is both timely and adds value to the ongoing discussion regarding the challenges associated with the pertinent theme of "DeepFake text detection", with a special focus on examining the boundaries of human detection. CCS Concepts: • Computing methodologies → Natural language processing ;

Journal of Artificial Intelligence and Technology, 2023

Even though Turnitin generates AI (Artificial Intelligence) writing detection reports, these AI reports shall not be used for punitive purposes as Turnitin AI reports accuracy is way below the 98% claimed by Turnitin, as revealed in this study. To assist professors, teachers, and content evaluation stakeholders in their strive to identify AI-generated material, this study examines the stylistic features of case study, business correspondence, and academic writing ChatGPT-4 generated responses by exploring sentence length, paragraph structure, word choice, mood, tense, voice, pronouns, keywords density, lexical density, lexical diversity, and reading ease. The study revealed that ChatGPT-4 case study generated responses are produced in paragraphs of 2 to 3 sentences of 16 to 18 words each. The sentences are mainly formed in imperative mood. The use of the second-person pronoun ‘you’ and the second-person possessive determiner ‘your’ is prevalent. Keywords and lexical density are rela...

International Journal for Educational Integrity

The proliferation of artificial intelligence (AI)-generated content, particularly from models like ChatGPT, presents potential challenges to academic integrity and raises concerns about plagiarism. This study investigates the capabilities of various AI content detection tools in discerning human and AI-authored content. Fifteen paragraphs each from ChatGPT Models 3.5 and 4 on the topic of cooling towers in the engineering process and five human-witten control responses were generated for evaluation. AI content detection tools developed by OpenAI, Writer, Copyleaks, GPTZero, and CrossPlag were used to evaluate these paragraphs. Findings reveal that the AI detection tools were more accurate in identifying content generated by GPT 3.5 than GPT 4. However, when applied to human-written control responses, the tools exhibited inconsistencies, producing false positives and uncertain classifications. This study underscores the need for further development and refinement of AI content detect...

2020

Massive digital disinformation is one of the main risks of modern society. Hundreds of models and linguistic analyses have been done to compare and contrast misleading and credible content online. However, most models do not remove the confounding factor of a topic or narrative when training, so the resulting models learn a clear topical separation for misleading versus credible content. We study the feasibility of using two strategies to disentangle the topic bias from the models to understand and explicitly measure linguistic and stylistic properties of content from misleading versus credible content. First, we develop conditional generative models to create news content that is characteristic of different credibility levels. We perform multi-dimensional evaluation of model performance on mimicking both the style and linguistic differences that distinguish news of different credibility using machine translation metrics and classification models. We show that even though generative...

Minds and Machines, 2022

This article contributes to the debate around the abilities of large language models such as GPT-3, dealing with: firstly, evaluating how well GPT does in the Turing Test, secondly the limits of such models, especially their tendency to generate falsehoods, and thirdly the social consequences of the problems these models have with truth-telling. We start by formalising the recently proposed notion of reversible questions, which Floridi & Chiriatti (2020) propose allow one to 'identify the nature of the source of their answers', as a probabilistic measure based on Item Response Theory from psychometrics. Following a critical assessment of the methodology which led previous scholars to dismiss GPT's abilities, we argue against claims that GPT-3 completely lacks semantic ability. Using ideas of compression, priming, distributional semantics and semantic webs we offer our own theory of the limits of large language models like GPT-3, and argue that GPT can competently engage in various semantic tasks. The real reason GPT's answers seem senseless being that truth-telling is not amongst them. We claim that these kinds of models cannot be forced into producing only true continuation, but rather to maximise their objective function they strategize to be plausible instead of truthful. This, we moreover claim, can hijack our intuitive capacity to evaluate the accuracy of its outputs. Finally, we show how this analysis predicts that a widespread adoption of language generators as tools for writing could result in permanent pollution of our informational ecosystem with massive amounts of very plausible but often untrue texts.

Journal of Responsible Technology

Melnyk, Y. B. (2025). Should we expect ethics from artificial intelligence: The case of ChatGPT text generation. International Journal of Science Annals, 8(1), 5–11. https://doi.org/10.26697/ijsa.2025.1.5 , 2025

Background and Aim of Study: Implementing artificial intelligence (AI) in various areas of human activity is an avalanche-like process. This situation has raised questions about the feasibility and regulation of AI use that require justification, particularly in the context of scientific research. The aim of the study: to identify the extent to which AI-based chatbots can meet ethical standards when analysing academic publications, given their current level of development. Material and Methods: The present study employed various theoretical methods, including analysis, synthesis, comparison, and generalisation of experimental studies and published data, to evaluate ChatGPT's capacity to adhere to fundamental ethical principles when analysing academic publications. Results: The present study characterised the possibilities of using AI for academic research and publication preparation. The paper analysed a case of text generation by ChatGPT and found that the information generated by the chatbot was falsified. This fact and other similar data described in publications indicate that ChatGPT has a policy to generate information on request at any cost. This completely disregards the reliability of such information, the copyright of its owners and the basic ethical standards for analysing academic publications established within the scientific community. Conclusions: It is becoming increasingly clear that AI and the various tools based on it will evolve rapidly and have qualities more and more similar to human intelligence. We believe the main danger lies in losing control of this AI development process. The rapid development of negative qualities in AI, such as selfishness, deceitfulness and aggressiveness, which were previously thought to be unique to humans, may in the future generate in AI the idea of achieving superiority over humans. In this context, lying and violating ethical standards when analysing academic publications seem like innocent, childish pranks at the early stages of AI development. The results are important in drawing the attention of developers, scientists, and the general public to the problems of AI and developing specific ethical standards, norms, and rules for its use in various fields.

2025

Deepseek and Qwen large language models (LLM) became popular at the beginning of 2025 with their low-cost and open-access LLM solutions. A company based in Hangzhou, Zhejiang, China, announced its new LLM, DeepSeek v3, in December 2024. Then, Alibaba released its AI model, Qwen 2.5 Max, on January 29, 2025. These tools, which are free, open-source, and have no specified or known query limit, have made a significant impact on the world. Deepseek and Qwen also have the potential to be used by many researchers and individuals around the world in academic writing and content creation. Therefore, it is important to determine the capacity of these new LLMs to generate high-quality academic content. This study aims to evaluate the academic writing performance of both Qwen 2.5 Max and DeepSeek v3 by comparing these models with popular systems such as ChatGPT, Gemini, Llama, Mistral, and Gemma. In this research, 40 articles on the topics of Digital Twin and Healthcare were used. The method of this study involves using generative AI tools to generate texts based on posed questions and paraphrased abstracts of these 40 articles. Then, the generated texts were evaluated through the plagiarism tool, AI detection tools, word count comparisons, semantic similarity tools and readability assessments. It was observed that plagiarism test result rates were generally higher for the paraphrased abstract texts and lower for the answers generated to the questions, but both were above acceptable levels. In the evaluations made with the AI detection tool, it was determined with high accuracy that all the generated texts were detected as AI-generated. In terms of the generated word count comparison, it was evaluated that all chatbots generated satisfactory amount of content. Semantic similarity tests show that the generated texts have high semantic overlap with the original texts. The results of the readability tests of the generated texts showed that the texts were insufficient in terms of readability.

International Journal for Educational Integrity

Academic plagiarism is a pressing concern in educational institutions. With the emergence of artificial intelligence (AI) chatbots, like ChatGPT, potential risks related to cheating and plagiarism have increased. This study aims to investigate the authenticity capabilities of ChatGPT models 3.5 and 4 in generating novel, coherent, and accurate responses that evade detection by text-matching software. The repeatability and reproducibility of both models were analyzed, showing that the generation of responses remains consistent. However, a two-sample t-test revealed insufficient evidence to support a statistically significant difference between the text-matching percentages of both models. Several strategies are proposed to address the challenges posed by AI integration in academic contexts; one probable solution is to promote self-transcendent ideals by implementing honor codes. It is also necessary to consider the restricted knowledge base of AI language models like GPT and address ...