The Grice Club

Welcome

The Grice Club

The club for all those whose members have no (other) club.

Is Grice the greatest philosopher that ever lived?

Search This Blog

Thursday, January 29, 2015

Why is the "Turing test" so bad?

Turing's imitation game is now referred to as "The Turing Test" and is taken by many (mostly computer scientists) to be a test of intelligence suitable for machines.

Either for that purpose, or for more philosophical purposes such as the philosophical analysis of the term "intelligent", its pretty bad (though it does, possibly because of its adoption by AI researchers and the ever growing reputation of Turing, engender huge amounts of philosophical debate).

Why is that?  How could a man of Turing's genius come up with such a naf test of intelligence?

The short answer is, I suspect, that he didn't.

One reason for doubting that he did is the disclaimer in the first paragraph, in which he decouples himself from the ordinary mental vocabulary, making clear that the imitation game does not give an account of those terms but rather provides a more definite substitute.

However, that's not my present concern, I want to suggest another reason to reconsider what Turing was trying to do, which possibly makes his enterprise more interesting philosophically than it might otherwise have been thought to be.

In that opening paragraph, Turing is not actually decoupling himself from the term "intelligent".  He is considering alternatives to the question "can machines think?".  So even if he got that right, it would still fall short of answering the question "could a machine be intelligent?".

My impression from the paper is that what Turing is trying to do is as follows.
He thinks that the brain, and perhaps also the mind, is a computational machine, and, having studied the limits of mechanical computation and invented (or discovered) the Universal Turing Machine (in the course of solving Hilbert's Entscheidungsproblem) he is convinced that any computational task can be accomplished by a suitably powerful electronic computer.
He is arguing that any intellectual feat which can be accomplished by a person can also be undertaken by computer, and it is that hypothesis which his experiment (the imitation game) is intended to test.   As it happens, humans are intelligent (often), and this is perhaps the one human capacity which people least expect a machine to emulate, so this becomes the headline and the title.

If Turing had been focussed on intelligence then he might have come up with something quite different, and perhaps even something which comes closer to a useful philosophical analysis of the term.  In this regard it may be noted that a common approach of scientists in making terms of ordinary language suitable for use in science is quantification, and of course this has been done for intelligence.  Quantity of intelligence is conventionally rendered as a numeric Intelligence Quotient (IQ), a measure introduced way back in 1912 by William Stern (according to Wikipedia).  It is an interesting question (on which I can offer no enlightenment) what philosophers of language have made of IQ as contribution to the analysis of the term "intelligent".

Interestingly, the usual manner in which IQ is measured effectively anticipates an important feature of the imitation game, which is that the identity (and race, and species, and even whether alive or inanimate) of the subject is concealed from the examiner.  An IQ test as usually applied is eminently suitable for use on machines.  So why did Turing not just argue in his paper that machines would eventually score highly in the standard IQ tests?

The answer is I think that what he was aiming to do was not just assert that machines could be intelligent, but to assert that intelligence suffices for all the mental capabilities exhibited by humans, even those very far removed from the skills tested by an IQ test.  Probably he was wrong, as a lifelong believer in AI, I nevertheless believe that intelligence, even if in some theoretical sense universal, will in practice be as varied in its practical effectiveness in machines as it is in humans.

This idea that Turing was only en passant providing a test of intelligence, and intending primarily to argue a somewhat different thesis is a new one for me (today!).  But my dislike of the Turing Test as defining AI is very long standing and I am pleased to say that I am now inclined to think Turing innocent in this matter, and place the blame more firmly on those Computer Scientists who take the emulation of human intelligence as the primary goal of their discipline.

I confess that I am remote from the field, and that my impression of how dominant this attitude is may be quite wrong.  It certainly is not universal, here's a quote which shows work on the other way of testing intelligence discussed here:

Researchers at the Department of Philosophy, Linguistics and Theory of Science at the University of Gothenburg, Sweden, have created a computer program that scored up to 150 on specific portions of an IQ test: identifying patterns in pictures and number sequences.

They seem to have bottled out on the more verbal problems, though natural language understanding is now "going mainstream" as Apple, Google, Microsoft, Amazon and others try to make their products responsive to verbal inputs. 

As far as analysis of the term "intelligence" is concerned, I am pretty confident that Grice would regard this kind of test, and use of the word "intelligence" to refer to what is assessed by it, as one small element in a much richer fabric, and might find that the grounds for any reductionist thesis (that intelligence in other forms might be somehow reducible to intelligence by this particular metric) are slender. 

Roger Jones


















2 comments:

  1. "As far as analysis of the term "intelligence" is concerned, I am pretty confident that Grice would regard this kind of test, and use of the word "intelligence" to refer to what is assessed by it, as one small element in a much richer fabric, and might find that the grounds for any reductionist thesis (that intelligence in other forms might be somehow reducible to intelligence by this particular metric) are slender."

    Beautiful.

    I would think Grice would start with 'linguistic botany', as he called it. Ways to describe people as intelligent. "Clever" perhaps. Only Mikes says that 'clever' is a dirty word, almost, in English -- never mind the ordinary English Grice was into! More in post, I hope.

    ReplyDelete
  2. If Turing (b. 1912) and Grice (b. 1913) would seriously be into analyzing 'clever', they should possibly consider Mikes.

    George Mikes,
    How Not to be Clever


    "You foreigners are so clever," said a lady to me some years ago."

    "First, thinking of the great amount of foreign idiots and half-wits I had had the honour of meeting, I considered this remark exaggerated but complimentary."

    "Since then I have learnt that it was far from it."

    "These few words expressed the lady's contempt and slight disgust for foreigners."

    "If you look up the word clever in any English dictionary, you will find that the dictionaries are out of date and mislead you on this point."

    "According to the Pocket Oxford Dictionary, for instance, the word means quick and neat in
    movement ... skillful, talented, ingenious."

    --- Nuttall is more Turingian. But cfr. Grice's reply to Austin, "I care a hoot what the dictionary says".

    Mikes:

    "Nuttall's Dictionary gives these meanings: dexterous, skillful, ingenious, quick or ready-witted, intelligent."

    "Intelligent" being Turing's favourite adjective.

    Mikes: "All nice adjectives, expressing valuable and estimable characteristics."

    "A modern Englishman, however, uses the word clever in the sense: shrewd, sly, furtive, surreptitious, treacherous, sneaking, crafty, un-English, un-Scottish, un-Welsh."

    "In England it is bad manners to be clever, to assert something confidently."

    "It may be your own personal view that two and two make four, but you must not state it in a self-assured way, because this is a democratic country and others
    may be of a different opinion."

    "A continental gentleman seeing a nice panorama may remark, 'This view reminds me of Ultrecht, where the peace treaty concluding the War of Spanish Succession was signed on the 11th April, 1713. The river there, however, recalls the Guadalquivir, which rises in the Sierra de Cazorla and flows south-west to the Atlantic Ocean and is 650 kilometres long. Oh rivers ... what did Pascal say about them? 'Les rivières sont les chemins qui marchent ...' "".

    Mikes:

    "This pompous, showing-off way of speaking is not permissible in England."

    "The Englishman looking at the same view would remain silent for two or three hours and think about how to put his profound feelings into words."

    "Then he would remark, 'It's pretty, isn't it?'".

    "An English professor of mathematics would say to his maid checking up the shopping list, 'I'm no good at arithmetic, I'm afraid. Please correct me, Jane, if I am wrong, but I believe the square root of 97344 is 312.'"

    "And about knowledge. An English girl, of course, would be able to learn just a little more about, say, geography."

    "But it is just not "chic" to know whether Budapest is the capital of Roumania, Hungary of Bulgaria."

    "And if she happens to know that Budapest is the capital of Roumania, she should at least be perplexed if Bucharest is mentioned suddenly."

    "It is so much nicer to ask, when someone speaks of Barbados, Banska Bystrica or Fiji: "Oh, those little islands ... Are they British?" (They usually
    are.)."

    ReplyDelete