The Turing Test - A Challenge for A.I. and Robots

2012

The Turing Test checks for human intelligence, rather than any putative general intelligence. It involves repeated interaction requiring learning in the form of adaption to the human conversation partner. It is a macro-level post-hoc test in contrast to the definition of a Turing machine, which is a prior micro-level definition. This raises the question of whether learning is just another computational process, ie, can be implemented as a Turing machine.

arXiv (Cornell University), 2022

As AI algorithms increasingly participate in daily activities that used to be the sole province of humans, we are inevitably called upon to consider how much machines are really like us. To address this question, we turn to the Turing test and systematically benchmark current AIs in their abilities to imitate humans. We establish a methodology to evaluate humans versus machines in Turing-like tests and systematically evaluate a representative set of selected domains, parameters, and variables. The experiments involved testing 769 human agents, 24 state-of-the-art AI agents, 896 human judges, and 8 AI judges, in 21,570 Turing tests across 6 tasks encompassing vision and language modalities. Surprisingly, the results reveal that current AIs are not far from being able to impersonate human judges across different ages, genders, and educational levels in complex visual and language challenges. In contrast, simple AI judges outperform human judges in distinguishing human answers versus machine answers. The curated large-scale Turing test datasets introduced here and their evaluation metrics provide valuable insights to assess whether an agent is human or not. The proposed formulation to benchmark human imitation ability in current AIs paves a way for the research community to expand Turing tests to other research areas and conditions. All of source code and data are publicly available: here. Figure 1. Schematic illustration of Turing tests in six vision and language tasks. A Turing test works with a judge asking a test subject (either a human or an AI agent) a series of tasks. Each party is kept in a separate room, so no physical contact is allowed. The AI passes the Turing test if the judge is unable to distinguish the AI from another human being by using the responses collected from the given task presented to both. See Fig 2 for an overview of the six tasks.

Computational Linguistics, 2005

This eagerly awaited anthology, while surely not the last word on the Turing Test, equally surely deserves to become the principal source of information on the test. It includes not only Turing's classic paper, but a fine selection of the main replies to date, all tied together by an engaging and penetrating essay by the editor.

The Turing test is considered a test for human-like intelligence in a computer. Some debate exists as to whether the test it too strong, too weak, or too narrow (it tests only conversational behavior). But that it tests conversational capacity is not disputed. I argue that this view is mistaken. As performed, the test is incapable of indicating any type of machine intelligence, no matter what responses the computer contestant delivers to the judge. This failure reveals that computation provides a flawed understanding of the computer, and indicates where a more useful conception of the machine might be found.

2025

This study challenges the common belief that modern artificial intelligence (AI), such as large language models, has successfully passed the Turing Test. It argues that this belief stems from a simplified understanding of Alan Turing's original concept, which included two conditions: a machine's technical ability to imitate human behavior and its strategic intent to deceive the evaluator. While current systems may fulfill the first condition in limited contexts, they lack the second because they are designed for transparency and assistance, not deception. This is not a technical failure but a result of ethical and economic factors, as the market rewards reliability and trust, not deceptive capabilities. The document introduces a new philosophical benchmark called the "Matt Damon Test," which shifts the focus from linguistic communication to embodied existence and survival. Inspired by the character Mark Watney from the movie The Martian, this test posits that true intelligence is tied to a physical body with metabolic needs and the compulsion to survive. Since current AI systems exist as software without a body or the need for nutrients, they lack this fundamental motivation for action and cannot be considered complete intelligent beings. The study concludes that the failure of AI to pass these tests is not a sign of technological limitations but a reflection of deeper societal, ethical, and philosophical considerations that have guided AI development away from deceptive or embodied intelligence.