What is Being Imitated? Revisiting the Idea of Intelligence in Turing Test

Deepa Shukla, Dr. George Kodimattam Joseph

Keywords: intelligence, turing test, imitation, artificial intelligence

TL;DR: The paper addresses two major structural challenges in the Turing test: a) indeterminacy of the object of imitation, and b) the variability of the interrogator.

Abstract:

Extended abstract:
What did Turing mean when he suggested that the best strategy might be to "imitate something other than a man” (Turing, 1950, pp. 435)? Did he foresee that a perfect imitation of human intelligence would eventually feel like a parody? Or the fact that deception as a criterion of imitation is problematic? Despite convincing many users about their intelligence and even consciousness, contemporary AI systems miserably fail the imitation game. After thorough analysis, it seems that the very conditions of the imitation game are flawed. The game assumes that linguistic indistinguishability, the machine's ability to imitate the conversational style of a normal human being, is a sufficient proxy for intelligence. Stated otherwise, a machine passes the Turing test if it successfully deceives a human interrogator. However, a closer look reveals that the imitation game suffers from deep structural uncertainties, which might also be the reason why contemporary systems often fail the game.

The primary issue that remains consistent throughout the article is the condition that passing the Turing test requires machines to ‘provide answers that would naturally be given by a man” (Turing, 1950, pp. 435). Let's ignore the gender bias here and assume that he meant ‘answers given by humans’. What kind of humans? It seems that Turing presupposes ‘human intelligence’ as a stable and identifiable target for imitation; however, he never specified the kind of intelligence to be imitated. The flaw lies in having a specific object of imitation. For instance, to imitate Mona Lisa is to have a specific object with well-defined features. However, this is not the case with human intelligence. Human intelligence is not a determinate object, but rather a fixed spectrum of performances shaped by context, ability, and circumstance. The test never specifies whether an average, expert-level or ideal human intelligence is to be imitated. This, in turn, removes any stable criteria of success and shifts the burden to the interrogator.

This shift marks another issue of the imitation game. If the success or failure of a machine depends on the interrogator’s knowledge, expectations, and interpretive abilities, then a machine may succeed with one interrogator and fail with another, suggesting that the test measures relative alignment instead of objective intelligence. The issue becomes more evident when applied to contemporary chatbots for emotional support. Chatbots, such as Replika, Webot, and Character.ai, are famous for imitating human emotional responses (Ta et al., 2020). However, the success or failure of this imitation significantly depends on the user's state. Machine generated imitations of human emotional responses are more likely to be experienced as convincing and meaningful when the user is in a vulnerable state. The point is that the system's ability to deceive the interlocutor depends heavily on the user's condition. When applied to Turing’s imitation game, a user's limited cognitive capacity may allow a system to pass the test.

In a nutshell, the paper briefly addresses two major structural challenges in the Turing test: a) indeterminacy of the object of imitation, and b) the variability of the interrogator. We argue that ‘human intelligence’ might not be the most appropriate benchmark for evaluation. As Turing suggested, machines may carry out something that ought to be described as thinking, but which is very different from what a man does. In this sense, we understand that by indicating ‘the best strategy for a machine may possibly be something other than the imitation of the man’ (Turing, 1950, pp. 435) Turing implicitly recognised the limitations of human-centred criteria and opened the possibility that artificial systems might instantiate a distinct form of intelligence. Accordingly, the paper proposes that the apparent failure of machines in the Imitation Game reflects not a lack of intelligence, but a mismatch between their mode of cognition and the unstable assumptions of the test, thereby calling for a broader reconceptualisation of intelligence beyond imitation.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 4