This aphorism, often used by Socrates, insists on the importance for an intelligent being to know himself. It is certainly useful to know whether one can make a particular action, what one wants to do, to follow the steps taken by his own thought, and so on. This is also useful for an artificial being, but it can have a knowledge of itself more exhaustive than we can ever do for ourselves: it can know its present state as completely as it wants. Sometimes, Artificial Cognition has entirely new possibilities.
We can know the position of our members, first with our vision, but mainly with proprioception, which gives us indications on the position, movement, and acceleration of our body parts. This is useful, although a part of this information is unconscious, it stops at the cerebellum. On the contrary, we have no similar information on which parts of our brain we are using: when I am writing or speaking, I am not aware that my Wernicke's area is very active. We will consider two situations where artificial beings may have a much more extensive knowledge of their state than ourselves: the knowledge of the state of all their components, and of all the bits of knowledge at their disposal.
For the present time, an artificial being is a set of programs running on a computer. For a complete knowledge of its current state, it must have access to the list of active subroutines. Each one called another subroutine down to the one that was running, when it has been interrupted so that it can observe itself. (In another post, I will examine several ways for interrupting a subroutine). One also needs to know the values of all the variables used in these subroutines. Note that one value may be a number, a name, or a complex structure such as the tree of all the moves considered for choosing the next move in a chess game.
CAIA is built in such a way that it can access the value of all these variables; for a living being, a similar capacity would be to know the state of any neuron, without perturbing it. Moreover, this neuron would not change its state, as long as we are observing it. Artificial beings have an extraordinary tool; we still have to create methods so that they can fully use this possibility.
Another limitation for human beings is the impossibility to know more than a small part of what they know. A large part of our knowledge is inaccessible, hidden in the mechanisms that we are using for each kind of activity. For instance, French speaking people will never write double consonants at the beginning or at the end of a word, although most of them do not know the rule that forbids it.
Some years ago, expert systems rightly arouse a wide interest. The idea was to ask an expert to give his expertise, and then to inset it into a program, which would have performances as good as those of the expert. Unfortunately, experts are often unable to describe their expertise: an expert knows what to do in any situation, but he does not know why he has chosen to do it. He can try to guess it, but he has not a direct access.
We have seen that CAIA uses knowledge as declarative as possible: in that way, it can access all the bits of knowledge that it can use. This is very useful: it can justify all the steps leading to a result, for instance, the proof of a theorem. Most humans are able to give such explanations, but it can also explain why it has chosen to perform any action, and why it has not tried another action. It can do that because it has also access to the bits of knowledge that choose what to do among what is allowed.
We are seriously handicapped on this point: teachers indicate the steps that lead to the solution; they rarely indicate why these steps have been chosen. In the same way, chess players indicate the moves that they have considered (this is an explanation), they are often unable to indicate why they have chosen to consider these moves. This would be a "meta-explanation", not an explanation of the solution, but an explanation of the method used for finding the solution. This last kind of decision usually depends on unconscious knowledge. For artificial beings, every action may be made conscious if necessary, because it can access every bit of knowledge that it has.
I have described elsewhere how this has been implemented in CAIA. These methods are well-known in Computer Science, they are used for debugging programs: the programmer needs to know everything that occurs when his program is running. My goal was that CAIA could have the same knowledge of its present state than the programmer has of the state of a program when he wants to find the cause of a bug. The methods for observing a program are the same, the only difference is that the individual who observes and analyses his observations is the program itself: one artificial being takes one human being's place.
Human consciousness: very useful, but could be improved
Consciousness allows us to know what we are thinking, and to think about it. We have already seen that this possibility is important, and we believe that we are very good in this activity. In fact, we are greatly better than animals, the best ones having only very limited capacities in this domain. However, this does not entail that we are so good: we could be like one-eyed among blinds.
What kind of information an intelligent being can have on the events that occur in his brain? First, it can have an information on the succession of steps that effectively occurred: I wanted to take my car, then I thought that there was a risk of ice, so I decided to take the train. We had access to the three steps that lead to the final decision. On the other hand, static information can also be useful: what parts of my brain were active during each of these steps? For this kind of information, we know absolutely nothing, if we are not using functional magnetic resonance imaging. We have no information on what happens in our brain, we do not even know that thinking is performed in the brain: Aristotle believed that the brain was a cooling mechanism for the blood!
Therefore, we will only examine the dynamic aspect of consciousness, which gives a partial trace of the steps that we have taken while we were thinking. A first limitation comes from the fact that this trace cannot be complete. If we are conscious of some events, there are also many events that we do not know. For instance, in the preceding example, we have thought of the ice, but why we have considered it, and not the possibility of traffic jams. In the same way, a chess player knows which moves he has considered while trying to choose his next move, but he knows almost nothing on the reason why he has considered only some of the legal moves, and not other ones. Many experiments have also shown that we have often a wrong idea of the reasons of a decision, like these people who could not believe that their decision partially depended on the position of the chosen cloth among other clothes. More seriously, when our subconscious has taken a decision, we do not always know it. Therefore, when we try to perform actions, which are against this decision, it usually succeeds to torpedo them. This explains why people, who are sensible, may have sometimes an inconsistent behavior.
Our brain is built in such a way that it can observe some of the events that happen in it, but not all of them: this is the reason of this limitation. The consciousness will never show the reason of some of our choices, because no mechanism can observe them. Only statistical methods suggest that an apparently secondary factor is actually essential. As our brain is essentially a parallel machine, it would have to observe many actions simultaneously; this would be very difficult to implement.
When we are trying to observe ourselves thinking, we disrupt how the functioning of our brain. This is a second limitation: we will never know what happens when we do not observe ourselves.
We cannot freeze a part of our brain, for quietly observing it, and then restart as if we had never stopped: this is a third limitation. This could be useful for analyzing the consequences of our recent actions, possibly to take the decision to change our plans, and finally to memorize the last steps. It is very difficult to create the trace of our actions: at the end of a thought process, we cannot remember all the steps that happened. It is possible to record a subject, trained to think aloud, while he is solving a problem. However, this constraint modifies his behavior; moreover, only what can be verbalized is produced. A trace will always be very incomplete because we cannot store the sequence of events, although our consciousness had known them.
To resume, consciousness shows only a part of our thoughts, and we cannot store more than a part of what was shown.
We will see that artificial beings are not restricted by these three limitations; moreover, they can statically examine their present state: with Artificial Cognition, one may have a super-consciousness. The difficulty is not to implement it, this has been made with CAIA, but to use its results efficiently. Indeed, a useful starting method in AI is to begin with an observation of our behavior, to implement a method similar to the one we are using, and to improve it when it shows imperfections. Unfortunately, this is impossible for finding how to use super-consciousness: we cannot model our behavior using a mechanism which is not available.
We are all convinced of the importance of consciousness: it is a tremendous difference between humans and animals. Therefore, the discovery of an efficient use of the super-consciousness will lead to a huge progress in the artificial beings’ performances, giving them capacities that we will never have.
Beware of the aliens
Many people seem to think that there is only one form of intelligence, ours, which could be more or less developed; for them, there cannot exist another intelligence that the kind created by evolution on earth, where we seem to be the most intelligent living beings. This belief was clearly shown when Pioneer 10 and 11 were launched. These space probes carry a plaque, as a bottle in the sea, for possible aliens. It indicates that there are humans on the third planet of our sun, that there are men and women, that they know the hyperfine transition of hydrogen, etc. Even for humans, it is not easy to understand what is in this plaque, and this could be done only by beings from a culture similar to ours. The authors of this message have implicitly supposed that a single path is possible for evolution, and that it would lead to individuals with a human-like intelligence .
Artificial Cognition shows us that it is not true: we can build beings with different intellectual characteristics. They can be very different from us for their speed, their memory, their consciousness, their senses, their actuators. This could lead to capacities that are unachievable for us. As the problem of constructing intelligent artificial beings has several solutions, I do not see why evolution will always lead to our intelligence, especially in environments where the temperature, pressure, heat, radiations, and so on, are completely different from the situation on earth.
Even on our earth, we have examples of other intelligent beings such as the societies of ants. They have remarkable performances: building nests, raising aphids, taking over other ant colonies and enslaving them, etc. Considered as a whole, an ant colony is intelligent, and this intelligence is created by the group. While we have about 85 billions of neurons in our brain, there may be millions of ants in a colony, and each one has 200,000 neurons: the ant colony may contain more neurons than our brain, but their organization is completely different. Each ant has limited cognitive capacities, but a group of so many individuals leads to an interesting behavior.
The creation of our intelligence by the evolutionary process strongly depended on the distribution of the resources we need, and on the methods used for creating and raising our offspring. The necessity to hunt and to gather food has led to improve some aspects of our perception, and of our problem solving capacities. In another environment, our capabilities would have evolved differently: for instance, if only the asexual reproduction existed, many aspects of our behavior would no longer be necessary, and evolution would have led to individuals unlike us. Another example, taken from a detail of the plaque: there is an arrow, its meaning is evident for us, who are coming from a hunter society, but aliens coming from different societies would not understand it.
Moreover, those who put this plaque on the Pioneers have also assumed that the aliens would be similar to scientists such as themselves, who usually behave in a civilized manner. However, we only have to look at the world history to find that man often misbehaved with other people raised in slightly different cultures. Even if the aliens had evolved like ourselves, the probability that they would be happy to meet other intelligent beings is very small.
Public reaction to the plaque was mostly positive. Curiously enough, most of the criticisms were on details such as: the humans were naked, the woman had a passive role, the woman’s genitalia were not depicted, etc. Only a few critics feared that it could lead to a disaster if some aliens would find and understand it. This idea was not widely held since this plaque was sent twice in the space, and later, a similar plaque, twice with Voyager.
I am personally aware of this problem because I am living with CAIA, the artificial being that I am creating. When I have obtained enough results from a version of CAIA, I am stopping it for good, it will never be active again: I will develop its successor. However, during its life, it has found some clever results, that many humans would be unable to understand. I do not feel guilty of “killing” it, its intelligence is too different from mine. Is it impossible that intelligent aliens view us in the same way? Is it impossible that, turning the tables, these aliens are artificial beings, created by living beings which have disappeared, leaving their world to their robots?
Luckily, it is highly unlikely that this plaque will go some day into the “hands” of aliens, and that they could understand it. However, if that happens, they will immediately destroy all life on earth with no more scruples than we have when we destroy an ant nest.
One reason why man is more intelligent than the animals
In many domains, some animals have excellent performances, such as the hunting techniques of most carnivores, the care of a hen for its chickens, the way a cat manipulates its master, and so on. Moreover, sometimes they have exceptional sensory organs such as the sharp eyesight of the raptors, the smell of the dog, the ultrasonic receptors of the bat. Some also have extraordinary physical performance: the speed of cheetahs and dolphins, the flight of birds. Although we are often inferior to animals, man conquered the world, and drastically transformed it. Why did we succeed? The answer is evident: we are more intelligent. Nevertheless, it is interesting to find which aspects of our intelligence are important for our superiority, so that we could give them to the artificial beings that we create.
Naturally, in several domains, we are better than animals, particularly with our capacity to communicate using language. I will insist here on the capacity of our brain to work at two levels. For instance, when we are solving a problem, we can try to execute some of the allowed actions, but we can also temporarily go at an upper level, where we no longer execute the actions that could solve the problem, but we examine its definition. In that phase, we find which actions could be efficient, those which could be a waste of time; we can also define new rules, which will help us to solve the problem more easily.
Let us consider the following problem: a snail is climbing up a 15 meters high mast. Each day, it is climbing up 3 meters, and each night it comes back 2 meters. When will it be at the top of the mast? Many people, even if they are not very clever, will answer 15 days. Unfortunately, this is false, but they have taken an extraordinary step: to find this result, they have looked at the formulation of the problem, and they have created a new rule: each day, the snail gains one meter. If the mast was one billion meters high, using this rule would lead to a drastic improvement compared with the method where we consider what happens during one billion days. The error was made because the new rule is misused: one would have to apply it at the evening of the first day, and not at the morning. However, it is remarkable that most humans find evident to create new rules, not by experience, but simply from the formulation of the problem.
This upper level is called the meta-level. The preceding example shows that human beings easily work at this level, where one thinks about the problem before applying its rules. In many situations, it is useful to work at two levels, particularly when we are monitoring the search for the solution of a problem: we foresee what could happen, then, after an action is performed, we look whether everything takes place as foreseen.
We also have to consider two levels when we examine the behavior of other people (or of our self). At the lower level someone thinks, and at the upper level another person (or the same one) thinks about what the first individual does when he is thinking. Psychologists call “metacognition” the capacity of modeling the other people (and also oneself). For instance, it is important to know what span we are able to jump, and animals are good at that. It is also important to know that repetition is useful for memorizing a text, and only man knows that. Apes, and particularly chimpanzees, have models of other chimpanzees, and of the humans that they often meet. However, their performances are not at our level. Dogs have more limited abilities: a guide dog for the blinds is extraordinarily devoted to its master. Unfortunately, it cannot foresee that its master will be hurt when they are walking under a 1.5 meters high scaffolding: it has not the capacity to put itself into the skin of a 1.8 meters high man. Naturally, it will avoid this place in the future, but it cannot avoid the first failure.
Consciousness also uses two levels: it allows us to access a part of the processes that take place in our brain. It is helpful to understand why we have taken a wrong decision, and to share our knowledge with other people since we are able to know a part of what we are knowing. Thanks to that, the master can directly give out a lot of knowledge to his pupils, who are not restricted to try to imitate him.
Too often, the work at the meta-level is done by the researcher who creates an AI system. Therefore, this severely restricts its adaptation to unforeseen situations: everything must be anticipated by the researcher, who has also to define the response of the system when they happen. As long as AI systems do not work at the meta-level, their intelligence will be very limited. Essentially, the source of their performances is the human analysis of the possible accidents or setbacks. As a guide dog for the blinds, these artificial beings are often unable to take a good decision in unexpected situations.
AI systems must not be sound
When a system is said to be sound, its results must be free from errors, defects, fallacies. In almost all the scientific domains, rigor is a highly desirable quality: one must not publish erroneous results, and the referees of a paper have to say whether this paper is sound. However, AI could still have an idiosyncratic position on this subject. Naturally, when a method finds correct results in a reasonable time, one must use it. Unfortunately, this is not always possible for three reasons: this would forbid to use methods that are often useful, or this could require centuries to get a result, or it is an impossible task.
Human beings often make mistakes in solving problems. When a journal publishes problems for its readers, it happens that the authors miss some solutions. Even Gauss, one of the greatest mathematical geniuses, had found only 74 of the 92 solutions of the eight queens puzzle: placing eight queens on a chessboard so that no two queens attack each other. A psychologist, who studied professional mathematicians, was surprised to find that they made many mistakes. The subjects were not surprised: for them, the important thing is not to avoid mistakes, but to find and correct them.
An AI system may also make mistakes, but this is not a reason to put it right into the trash: finding at least one solution is often useful, even if one does not get all of them. Indeed, the commonest error is to miss solutions. It is easy to check that a solution is correct: one has only to verify that all the constraints are satisfied by a candidate solution. It is much more difficult to be sure that no solution has been bypassed.
In a combinatorial program, one considers all the combinations of values of the variables, and one keeps those that make all the constraints true. These programs are rather simple, and it is reasonably possible to avoid bugs. However, even here, one can miss solutions. For example, one of the first programs that created all the winning positions for a particular set of chess pieces generated several millions of such positions. Once it was shown to grandmaster Timman, he found that a winning position had been forgotten. Naturally, the bug was removed, all in all half a dozen solutions were missing. At the same time, even the results of the erroneous program were useful: the probability that one comes across one of the missing winning positions is very low. A program using this database would have very good results.
However, combinatorial methods may require too much time, and they cannot be used when there is an infinity of possible values for a variable. For finding a solution, one can exchange rigor against time. Therefore, knowledge is used for eliminating many attempts, which would not lead to a solution. Unfortunately, if we wrongly eliminate an attempt, we will miss a solution. And when there is a lot of such knowledge, it is likely that a part of it is incorrect.
Finally, several mathematicians have proven that we cannot hope to prove everything: any complex system, either sometimes produces false statements, or else there are true statements that it will never prove. This limitation is very serious: this is not because we are not enough intelligent, this is an impossibility; even future super-intelligent artificial beings will also be restricted. It is interesting to examine the proofs of these results; this is not so difficult, it exists a very simple proof found by Smullyan. The reason is always: when a system is powerful enough to consider its own behavior, and when its actions depend on this observation, it is restricted in that way. Therefore, it will not be sound: either it finds fallacies, or it misses results.
Systems such as CAIA create a large part of their knowledge, they use a lot of knowledge for eliminating many attempts, they analyze their behavior to find bugs, etc. These are the characteristics that can sometimes lead to unsound results.
I believe that an AI system must not be sound. If it is sound, it is not ambitious enough: it contains too much human intelligence, and not enough artificial intelligence. Human beings are not sound, such as this lady of the XVIIIth century, who did not believe in ghosts, but was afraid of them. Artificial beings have also to bear the burden of these restrictions: the clever they are, the more unsound they will be. Naturally, we must and can manage to remove as many errors as possible, but we cannot hope to remove all of them.
Hard tasks may be easier than easy tasks
What we are easily doing may be very difficult for an AI system. Conversely, it can easily perform tasks that are very difficult for human beings. The reason is that, during millions of years, evolution made us very expert for some activities such as perception, but has not adapted us to many activities that are not useful for feeding or breeding. Seeing where a soccer ball is is not evident for an artificial player when this ball has not been specially colored so that it clearly appears in the background. Our ancestors, hunters or gatherers, needed excellent perceptual abilities for surviving; we are not even better than animals in that domain, evolution considerably increased the potential of all living beings. On the contrary, living beings are not so good for reasoning: most animals cannot make simple deductions. Among the best ones, apes are able to see that it is necessary to move a box under bananas fastened to the ceiling so that they could take them; this is one of the most difficult problems that they can solve. Man is incomparably more successful than apes for reasoning; however, we have severe limitations. Many people do not find easily whether there are more animals that are not ducks, or more animals that are not birds. We overestimate our performances in this domain because we can only compare ourselves with the animals. Moreover, we try to avoid as much as possible problems that we have great difficulties to solve.
The Spring 2012 issue of the AI magazine describes a competition for chess-playing robots. Since several years, some programs are very strong, better than the best human players: they accept to play against programs with human friendly restrictions, for instance, the program can only use a part of the endings database, which now contain all the winning positions up to seven pieces. In this competition, the difficulty was not to find a good move, but to perceive the board seen by a camera, and to play effectively this move. The system does not receive the last move via a keyboard, and it does not display its own move on a monitor. It only sees a real chess position with the usual figures of the chess pieces. Afterwards, once it has found its move, it really plays it, first removing a possible captured enemy man, then taking its own piece, and moving it to its new square.
In this experiment, the robot is not very skillful. For evaluating the competitors, points were awarded for making a move in less than 5 minutes, points were subtracted for knocking over a piece, or failing to place a piece fully within its arrival square. To keep the game short, only 10 moves were played by each side. Moreover, some participants were helped by defining an environment facilitating the perception of the scene: one robot (not the winner) was playing with blue pieces against yellow ones, on a white and green chessboard. Although the participants have realized exceptionally well designed robots, the perceptual and motor limitations of the robots, compared with the remarkable quality of chess programs, clearly show the difference of the results among applications common to man and machine. For an artificial being, it is easier to find a winning move against the human world chess champion than to play this move on the chessboard physically!
Naturally, we must carry on with the realization of applications using our perceptual and motor abilities: such robots can be very helpful, and in some cases they can completely replace us for dangerous or boring applications. For instance, robots that can drive a car will be interesting when they will become skillful enough. However, implementing an AI system that solves a problem, which is extraordinary difficult for us, could be no more difficult to realize than building a system solving another problem that every man, and in some cases even animals, can solve easily.
Although a mathematician has recently proven that using mathematics could help finding a bride, this is an exceptional situation: it is unlikely that outstanding mathematicians have better rates of survival and reproduction than other humans. Our capacities in this domain are only a by-product of capacities that has been developed for other goals. The best mathematicians are very good in a relative scale, compared to other human beings, but they may be very low in an absolute scale, when one considers all the entities that could some day perform mathematical activities. Perhaps, they are only one-eyed among blinds, maybe very far from what could be done by artificial beings. I remember how chess players were laughing in the 1970s when they were looking at the moves played by the best programs at this time. They have completely changed their opinion on this point, even the grandmasters now use programs for preparing their matches.
The results obtained in AI show that AI research is extremely difficult for humans. For this reason, I am developing CAIA, an artificial Artificial Intelligence researcher. This is certainly a very difficult task, but evolution has not built us for being good AI researchers. Therefore, this problem is perhaps not so difficult that we could believe at first sight. When I have managed to improve CAIA so that it takes over from me for some particular task, it always performs better than myself.
Stop programming !
Why am I asking to stop programming when I am advocating the experimentation of large systems by AI researchers? In fact, I recommend using a computer, but without programming it.
Indeed, programming is a complex task, and man is not very good at doing it, as we can see from the number of our bugs, and the delays, mistakes, and costs that they entail. When a computer works on an AI problem, it needs a huge amount of knowledge, but we must not give it inside programs. This knowledge is often unknown in the beginning, we have to experiment the system so that we can improve and complete it. Unfortunately, it is very difficult to modify a program, and we add a lot of new bugs doing it. To lessen this difficulty, it is better to separate knowledge from the way to use it, we must not give it in a procedural form as it is in a computer program: we give knowledge in a declarative form, which does not include how to use it. The drawback is that we need a general program that can use declarative knowledge.
Let us first examine what is declarative knowledge; the following sentence, taken from a French grammar, is declarative:
Articles agree in gender and number with the noun they determine.
This does not indicate whether one must verify this agreement when one finds an article, or a name, or at the end of the processing of a sentence, etc. It does not tell what one does when there is a disagreement, or if it is used when one processes a text in order to understand it, or when one writes a text, or in both cases. One could even decide not to use this rule in some situations, for instance, when parsing a text. An advantage of using declarative knowledge is that it is easy to modify it: if one wants the analogous rule for English, it is enough to remove the gender agreement, for Greek, to add “case”, and for Latin, where there are no articles, to remove it.
Some could say that the problem is not solved, we create a new problem, more difficult than the initial one: writing a program that can use declarative knowledge. However, we win because only one program is needed, it will be able to use declarative knowledge in any domain. Moreover, this reflectivity leads to another advantage: one can bootstrap the use of declarative knowledge, the knowledge necessary for using declarative knowledge must also be given in a declarative form.
One factor makes easier this bootstrap: knowledge may be more or less declarative, there is a large gap between purely procedural knowledge (such as it is in computer programs) and purely declarative knowledge (such as in the preceding grammar rule). When knowledge is more declarative, it is easier to create and modify it. The bootstrap progresses in increasing the declarativity of the pieces of knowledge used for solving our problem, and also the declarativity of the pieces that are necessary for using declarative knowledge. Let us give a less declarative example of the preceding grammar rule:
When one parses a text, and one finds a noun, one looks for its article, and one checks that both genders and numbers are the same.
We have mixed grammatical knowledge with indications on how to use it. This rule is less general than the initial one: it will not be used for generating a text.
For bootstrapping the use of declarative knowledge, I began with a program that could manage to use rather procedural knowledge, with only a few declarative aspects. This program was simple: it is easy to use knowledge, when it is given in a form similar to a programming language. With this program, the initial knowledge was transformed into a program. Since that time, I only had to increase the declarativity of knowledge, the old program creates a new program, which can use this new version. In that way, it is easier to give knowledge: using it, the system can accept knowledge more and more declarative. CAIA has written the 470,000 lines of C that make up CAIA for the present time, none has been written by myself. On the other hand, every bit of knowledge is now given in a formalism much more convenient that 25 years ago.
In that way, besides the main bootstrap where I am realizing an artificial AI researcher that will help me to advance AI, a “small” bootstrap makes easier the realization of this artificial researcher: its knowledge is given in a more and more declarative formalism.
Naturally, we must have programs, but they have to be written by the AI system itself. During the development of the AI bootstrap, each of the participants, myself and CAIA, has to do the tasks that he does the best. Artificial beings are much better than ourselves for programming: they are faster, and they are making less bugs. Contracting out the programming activities gives us more time for doing what we are still doing better than artificial beings: finding new knowledge in order to improve them.
Adolphe Pégoud, a model for AI researchers
We have just celebrated the centenary of two achievements of the aviator Adolphe Pégoud: the first parachute jump from a plane, and the invention of aerobatics. Curiously enough, both are strongly connected; it is interesting to understand how the first achievement led to the second one.
Parachute jumps were already made from balloons, but never from a plane. Pégoud thought that it could be useful for leaving a broken-down plane. The other pilots thought that he was crazy to try such a dangerous and pointless experiment. Moreover, as most planes had room for only the pilot, the plane would be lost after the pilot left the plane. Everybody, including Pégoud, did not think much about the future of the plane, but they believed that it would immediately crash when the pilot would have jumped. Pégoud had chosen an old plane which was expendable. While he was coming down under his parachute, Pégoud looked at his plane, and he was very surprised by its behavior. Instead of immediately crashing, it made many curious maneuvers, for instance, flying upside down, or looping the loop, and this did not lead to a disaster: it carried on with another aerobatics. Pégoud immediately understood the interest of this experience: if the plane could do these figures without a pilot, it could do them with a pilot. Therefore, after being solidly tied up to his plane, he imitated his preceding plane, and he was the first human to fly upside down; a little later, he was looping the loop.
Pégoud realized that a plane could “discover” new maneuvers when it was left autonomous, and he was able to take advantage of this capacity. We, AI researchers, must also imitate Pégoud, leaving our systems free to make choices, to fly around, in situations where we have not given them specific directives. Then, we analyze what it has done, and we possibly find new ideas that we will include in our future systems.
Personally, I have had such an experiment, and it gave me the idea about the importance of a direction of research that I am trying to develop since I am working in AI. In 1960, I began working on a thesis in AI. My goal was to realize a program that had some of the capacities of a mathematician: it received the formulation of a theory (axioms and derivation rules), and it had to find and prove theorems in this theory. Although it could try to prove a conjecture that I gave it, it usually started its work without knowing any theorem of this theory. As Pégoud’s plane, it had no master, it was free to choose which deductions it would try: I had no idea of the directions that it would take.
One day, as I was analyzing the results found by the second version of this system, I was surprised to see that it had found a proof of a theorem different from the proof that I knew, which was given in the logic manuals. It happened that, for proving theorem TA, it did not use another theorem TB, whereas TB was essential for the usual proof; the new proof was shorter and simpler. I tried to understand the reasons behind this success; I discovered that the system, left to itself, had behaved as if it had proven and used a meta-theorem (or theorem on the theory) that allowed to find the proof without using theorem TB: the system bypassed it. After this experiment, as Pégoud, I took over the controls, and I realized a third version of my system, which systematically imitated what I had observed: the system was now able to prove meta-theorems in addition to theorems. It could study the formulation of the theory, and not only using the derivation rules with the already found theorems. This new version had much better results: it proved more theorems, the demonstrations were more elegant, and they were found more quickly.
Since that experiment “à la Pégoud”, I am convinced that an AI system must be able to work at the meta-level if we want it to be both effective and general. In doing so, it can fly over the problems, and discover shortcuts or new methods: it is easier to find the exit of a labyrinth when one is above it.
Such discoveries are possible only when we let our systems free to choose what they will do. If we want to bootstrap AI, we have to be helped by new ideas coming from observations on their behavior, while we are parachuting down after leaving them alone.
A new science: Artificial Cognition
Many scientists in Cognitive Science study the cognition of living beings, human or animal ones. However, some capacities of artificial beings are completely different from those of the living ones. Therefore, a new science, Artificial Cognition, will have the task to examine the behavior of artificial beings.
For Cognitive Science, living beings exist; we want to understand the reasons behind their behavior, and their capacities in various situations. We want to know how they memorize, remember, solve problems, take decisions, and so on. We observe them, we use medical imaging to detect what parts of a brain are active when the subject perform a task. One also devises ingenuous experiments that will show how the subject manages to solve a problem cleverly chosen. Naturally, we only study behaviors that exist for at least one kind of living beings.
The situation is different for Artificial Cognition: the goal is to build artificial beings rather than to observe them. Normally, we write computer programs, or we give knowledge to existing systems, and we try to obtain an interesting behavior. For that we usually utilize ordinary computers, but we can also build specialized machines, and this will be probably more frequently the case in the future. Living beings depend on mechanisms created by evolution, which uses mainly a remarkable element, the neuron. They may have extraordinary capacities for adaptation: we can learn to build houses, to write books, to grow plants, etc. Unfortunately, we have also limitations: we cannot increase the size of our working memory to more than about 7 elements; we can only use auxiliary memories such that a paper sheet. They are useful, but not as efficient as our internal memory. We can no more increase the possibilities of our consciousness, many mechanisms of our brain will always be hidden when we are thinking. This is a very serious restriction: consciousness is essential for learning, for monitoring our actions, for finding the reason of our mistakes, and so on.
On the contrary, in Artificial Cognition, we are not restricted to the neuron, we can build the mechanisms that we have defined. This possibility does not exist in the usual Cognitive Science: nature has built the beings that we want to study. In Artificial Cognition, we put ourselves in the place of evolution, which worked during billions of years on zillions of subjects. It succeeded in creating living beings, often remarkably adapted to their background. However, nobody is particularly well adapted to the artificial environments that man created, such as solving mathematical problems, playing chess, managing a large company, etc. As we have invented many of these activities, we have chosen them so that we can have reasonable performance in these domains, using capacities that evolution gave us for different goals such as hunting game for food. At the start, when on tries to build a new system, we are inspired by our methods, such as they have been discovered by Cognitive Science scientists. In doing so, we are using only a small part of the possibilities of Artificial Cognition, we must also utilize all the possibilities of computers, even those that we cannot have. Artificial beings will have much better performances than us when they use not only all of our methods, but also many other methods that we cannot use. We are unable to use many useful methods because we have not enough neurons, because they are not wired in the necessary way; it may be also simply because our neurons have intrinsic limitations, for instance, they do not allow to load new knowledge in our brain easily. Perhaps, there are capacities that evolution did not give us either because they were not useful for our ancestors, or because there are jumps that evolution cannot make.
The methodology and the potentiality of the usual Cognitive Science and of Artificial Cognition are very different. We are not limited to the existing beings, but it is very difficult to build new beings. However, there is a strong tie between these two sciences: building an artificial being is defining a model. If it behaves as living beings do, this model will give an excellent description of the mechanisms that Cognitive Science wants to find. On the other hand, when we want to build an artificial being, the first thing to do is always to start with the implementation of the methods that are used by living beings. Nevertheless, we have to progress from this starting point, and we will arrive perhaps some day to build artificial beings that will be able to achieve tasks extremely difficult for us. For instance, we will see artificial beings capable of building other artificial beings more effective than themselves.
The future of AI is the Good Old Fashioned Artificial Intelligence
AI researchers have various goals: many are mainly interested in studying some aspects of intelligence, and want to build rigorous systems, often based on a sophisticated mathematical analysis. One characteristic of this approach is to divide AI in many sub-domains such as belief revision, collective decision making, temporal reasoning, planning, and so on. However, other researchers want to model human behavior, while I belong to a third category, which only wants to create systems solving as many problems as possible in the most efficient way.
At the beginning of AI, the supporters of the last approach were the majority but, with the passing years, they have become such a minority that many do not understand the interest of this approach, which they judge unrealistic, and even non scientific. Some present AI researchers look with condescension at those who are still working on these ideas, and they are speaking of the Good Old Fashioned Artificial Intelligence. Very funny, the acronym is almost Goofy! However, one can be arrogant when one has obtained excellent results, which is certainly not the case: AI has not yet changed the life of human beings. It is better to think that there are several approaches, and that all of them must be developed as long as one of them has not succeeded in making a significant breakthrough.
In my approach, we experiment very large systems, using a lot of knowledge. It is very difficult to foresee what results such a system will obtain: its behavior is unpredictable. Usually, one has unpleasant surprises, we have not at all the excellent results that we expected. Therefore, we have to understand why it goes wrong, and to correct the initial knowledge. During this analysis, we may find new concepts that will enable us to improve our methods. Finally, almost nothing of the first system is still present after this succession of modifications. For this reason, a paper where a researcher presents what he wants to do at the start of his research is not very interesting, the final system will be too different from the initial one. The interest of the first inspiration is that it is necessary for starting the process of improving this succession of systems. Only the last version is really useful. We have to start in a promising direction, and to describe only what we have built at the end.
This method has many drawbacks for the scientifically correct approach. First, we cannot publish many papers: we must wait to have a system that has interesting results, and that may require several years. Moreover, it is almost impossible to describe a very large system with enough precision, so that another researcher could reproduce it. Naturally, it is always possible to take the program, and check that it gets the same results but, if so, one does not really understand how it works. For being convinced of the interest of a system, a scientist wants to create it again. Unfortunately, that requires a lot of time since these systems use a lot of knowledge. Moreover, they are so complex that it is impossible to give a complete description, too many minor details are important for the success of the new system. For instance, CAIA includes more than 10,000 rules, and a twenty pages paper may be necessary for explaining only fifty of them. I could remake Laurière’s ALICE because I could question him about important choices which he had not the place to include into the 200 pages of his thesis.
We can understand than many researchers reluctantly look for an approach that has not a beautiful mathematical rigor. Unfortunately, it is not evident that mathematics are appropriate for AI, rigor is often too costly in computer time. If a system using a perfectly rigorous method can solve a problem, that is the best solution. For instance, that is the case for Chess endings with at most seven pieces. However, it does not seem that it is always possible, theoretical results prove that, for some problems, the computer time necessary for the fastest solution, increases dramatically with the size of this problem.
For the most complex problems, we must trade perfection against speed, and realize systems that solve many, but not all, problems in a reasonable time. It seems that such systems have to use a huge amount of knowledge. As they are very large, it is impossible to be sure that they never make mistakes. However, it is better to have a system that correctly solves many problems, and makes a few mistakes, than a system that fails to solve most problems because they would require too must time. After all, human beings often make mistakes; this does not prevent us to make sometimes good decisions.