Why is the "Turing test" so bad?

Turing's imitation game is now referred to as "The Turing Test" and is taken by many (mostly computer scientists) to be a test of intelligence suitable for machines.

Either for that purpose, or for more philosophical purposes such as the philosophical analysis of the term "intelligent", its pretty bad (though it does, possibly because of its adoption by AI researchers and the ever growing reputation of Turing, engender huge amounts of philosophical debate).

Why is that?  How could a man of Turing's genius come up with such a naf test of intelligence?

The short answer is, I suspect, that he didn't.

One reason for doubting that he did is the disclaimer in the first paragraph, in which he decouples himself from the ordinary mental vocabulary, making clear that the imitation game does not give an account of those terms but rather provides a more definite substitute.

However, that's not my present concern, I want to suggest another reason to reconsider what Turing was trying to do, which possibly makes his enterprise more interesting philosophically than it might otherwise have been thought to be.

In that opening paragraph, Turing is not actually decoupling himself from the term "intelligent".  He is considering alternatives to the question "can machines think?".  So even if he got that right, it would still fall short of answering the question "could a machine be intelligent?".

My impression from the paper is that what Turing is trying to do is as follows.
He thinks that the brain, and perhaps also the mind, is a computational machine, and, having studied the limits of mechanical computation and invented (or discovered) the Universal Turing Machine (in the course of solving Hilbert's Entscheidungsproblem) he is convinced that any computational task can be accomplished by a suitably powerful electronic computer.
He is arguing that any intellectual feat which can be accomplished by a person can also be undertaken by computer, and it is that hypothesis which his experiment (the imitation game) is intended to test.   As it happens, humans are intelligent (often), and this is perhaps the one human capacity which people least expect a machine to emulate, so this becomes the headline and the title.

If Turing had been focussed on intelligence then he might have come up with something quite different, and perhaps even something which comes closer to a useful philosophical analysis of the term.  In this regard it may be noted that a common approach of scientists in making terms of ordinary language suitable for use in science is quantification, and of course this has been done for intelligence.  Quantity of intelligence is conventionally rendered as a numeric Intelligence Quotient (IQ), a measure introduced way back in 1912 by William Stern (according to Wikipedia).  It is an interesting question (on which I can offer no enlightenment) what philosophers of language have made of IQ as contribution to the analysis of the term "intelligent".

Interestingly, the usual manner in which IQ is measured effectively anticipates an important feature of the imitation game, which is that the identity (and race, and species, and even whether alive or inanimate) of the subject is concealed from the examiner.  An IQ test as usually applied is eminently suitable for use on machines.  So why did Turing not just argue in his paper that machines would eventually score highly in the standard IQ tests?

The answer is I think that what he was aiming to do was not just assert that machines could be intelligent, but to assert that intelligence suffices for all the mental capabilities exhibited by humans, even those very far removed from the skills tested by an IQ test.  Probably he was wrong, as a lifelong believer in AI, I nevertheless believe that intelligence, even if in some theoretical sense universal, will in practice be as varied in its practical effectiveness in machines as it is in humans.

This idea that Turing was only en passant providing a test of intelligence, and intending primarily to argue a somewhat different thesis is a new one for me (today!).  But my dislike of the Turing Test as defining AI is very long standing and I am pleased to say that I am now inclined to think Turing innocent in this matter, and place the blame more firmly on those Computer Scientists who take the emulation of human intelligence as the primary goal of their discipline.

I confess that I am remote from the field, and that my impression of how dominant this attitude is may be quite wrong.  It certainly is not universal, here's a quote which shows work on the other way of testing intelligence discussed here:

Researchers at the Department of Philosophy, Linguistics and Theory of Science at the University of Gothenburg, Sweden, have created a computer program that scored up to 150 on specific portions of an IQ test: identifying patterns in pictures and number sequences.

They seem to have bottled out on the more verbal problems, though natural language understanding is now "going mainstream" as Apple, Google, Microsoft, Amazon and others try to make their products responsive to verbal inputs. 

As far as analysis of the term "intelligence" is concerned, I am pretty confident that Grice would regard this kind of test, and use of the word "intelligence" to refer to what is assessed by it, as one small element in a much richer fabric, and might find that the grounds for any reductionist thesis (that intelligence in other forms might be somehow reducible to intelligence by this particular metric) are slender. 

Roger Jones