Why am I asking to stop programming when I am advocating the experimentation of large systems by AI researchers? In fact, I recommend using a computer, but without programming it.
Indeed, programming is a complex task, and man is not very good at doing it, as we can see from the number of our bugs, and the delays, mistakes, and costs that they entail. When a computer works on an AI problem, it needs a huge amount of knowledge, but we must not give it inside programs. This knowledge is often unknown in the beginning, we have to experiment the system so that we can improve and complete it. Unfortunately, it is very difficult to modify a program, and we add a lot of new bugs doing it. To lessen this difficulty, it is better to separate knowledge from the way to use it, we must not give it in a procedural form as it is in a computer program: we give knowledge in a declarative form, which does not include how to use it. The drawback is that we need a general program that can use declarative knowledge.
Let us first examine what is declarative knowledge; the following sentence, taken from a French grammar, is declarative:
Articles agree in gender and number with the noun they determine.
This does not indicate whether one must verify this agreement when one finds an article, or a name, or at the end of the processing of a sentence, etc. It does not tell what one does when there is a disagreement, or if it is used when one processes a text in order to understand it, or when one writes a text, or in both cases. One could even decide not to use this rule in some situations, for instance, when parsing a text. An advantage of using declarative knowledge is that it is easy to modify it: if one wants the analogous rule for English, it is enough to remove the gender agreement, for Greek, to add “case”, and for Latin, where there are no articles, to remove it.
Some could say that the problem is not solved, we create a new problem, more difficult than the initial one: writing a program that can use declarative knowledge. However, we win because only one program is needed, it will be able to use declarative knowledge in any domain. Moreover, this reflectivity leads to another advantage: one can bootstrap the use of declarative knowledge, the knowledge necessary for using declarative knowledge must also be given in a declarative form.
One factor makes easier this bootstrap: knowledge may be more or less declarative, there is a large gap between purely procedural knowledge (such as it is in computer programs) and purely declarative knowledge (such as in the preceding grammar rule). When knowledge is more declarative, it is easier to create and modify it. The bootstrap progresses in increasing the declarativity of the pieces of knowledge used for solving our problem, and also the declarativity of the pieces that are necessary for using declarative knowledge. Let us give a less declarative example of the preceding grammar rule:
When one parses a text, and one finds a noun, one looks for its article, and one checks that both genders and numbers are the same.
We have mixed grammatical knowledge with indications on how to use it. This rule is less general than the initial one: it will not be used for generating a text.
For bootstrapping the use of declarative knowledge, I began with a program that could manage to use rather procedural knowledge, with only a few declarative aspects. This program was simple: it is easy to use knowledge, when it is given in a form similar to a programming language. With this program, the initial knowledge was transformed into a program. Since that time, I only had to increase the declarativity of knowledge, the old program creates a new program, which can use this new version. In that way, it is easier to give knowledge: using it, the system can accept knowledge more and more declarative. CAIA has written the 470,000 lines of C that make up CAIA for the present time, none has been written by myself. On the other hand, every bit of knowledge is now given in a formalism much more convenient that 25 years ago.
In that way, besides the main bootstrap where I am realizing an artificial AI researcher that will help me to advance AI, a “small” bootstrap makes easier the realization of this artificial researcher: its knowledge is given in a more and more declarative formalism.
Naturally, we must have programs, but they have to be written by the AI system itself. During the development of the AI bootstrap, each of the participants, myself and CAIA, has to do the tasks that he does the best. Artificial beings are much better than ourselves for programming: they are faster, and they are making less bugs. Contracting out the programming activities gives us more time for doing what we are still doing better than artificial beings: finding new knowledge in order to improve them.
How did you count the number of source code lines in CAIA?
In the version pi of CAIA that you gave me in august 2013, by counting with wc *.c | sort -n | tail I’m getting 518711 lines. Or has CAIA shrinken since then? Why?
In your version, each program has 5 more lines for the GNU General Public License.
With J.Pitrat’s explicit permission and help, I have put the source archive of the version qb (dated february 05, 2014) of CAIA available (free software under GPLv3 license) on http://starynkevitch.net/Basile/caia-qb-05feb2014.tgz – notice that the system has a crude user interface. You may want to type L# (to get some very declarative math knowledge) or L EDITE (to get the rules for the command line interface).
I have been a believer in this « Declarative is better than Procedural in AI » faith for a long time. I am not sure I am anymore. Or at least I wonder what « declarative » actually means.
First of all, I believe that we prefer declarative things because they are closer to the way we think, whereas computer procedures, in « machine language », are far away. So, we are not solving a problem for the machine, but for us. Second, declarative knowledge is supposed to be easier or more general to proceed than procedural one, especially for meta-knowledge. It is true that meta-knowledge, i.e. knowledge that works on knowledge, must work on a representation of the pieces of knowledge K it works on, and that if this representation of K is too far away from what K does (its « phenotype ») then again it is very difficult for us to write meta-knowledge: the organ must not be too far apart from the function for us to understand their relationship, and manage to make something meaningful (for us) when we design or change the system. So again, this is our limitation, not the system’s.
However, in biology, the representation of a protein (which is defined, up to some transcription, through a gene) is far apart, for us, from its function. It’s the whole issue behind synthetic biology – that is a kind of « AI in biology ».
So, what does declarativeness really means, if we mean it as a quality of the representation the (meta-) system works on? I can see one general definition: that it makes evolution possible, not only in formal way, but also « efficient » enough to enable viability of the system which embeds it. There is probably such quality in the way life has implemented genes, proteins and their expression – though it seems still unclear to us humans today.
Now, coming back to the bootstrap, evolution is mostly directed by its designer, at least during its first steps. And so we come back to the « true » notion of declarativeness: making things easier for the « bootstrapper ».
Let me add an intuitive thought. In mathematics, when one studies dynamic systems, or in other scientific disciplines that use these mathematics, such as physics to finance, the first-order object that one considers is a function. Then one can define a function by giving some procedure to compute it. Or one can use a formula. However, this is not very general and not adapted to what we have at hand. The most general way is to define the constraints, or conditions, or laws, whatever you call it, that the function must fulfill or obey. It turns out that the formulation of such constraints uses the notion of function; one also says that they mention operators – which are functions – predicates – which are functions having booleans as value – or functionals – which are functions of functions, and so functions, if the notion of function is defined with sufficient generality. For example, a system characterized by state variable p (i.e., a function from time to the state space, which can be of any dimension, even a function space) will minimize some action H(p, dp/dt).
An artificial neuron network is a program for solving (learning phase) through some special kind of procedure generation (actually, generating the weights of the neurons, which are the variable part of the generated procedure), and for executing this procedure.
CAIA is a system for solving combinatorial constraints, which are analog to dynamic constraints when the state space is finite or at least discrete – though I can see no limitation in principle to extend CAIA to the continuous case, this si what Jean-Louis Laurière did in RABBIT, or what e.g. people from the MODELICA community are doing, even if they do not think that they are doing AI.
So, all those systems seem to rely on analog concepts, and can possibly be the basis for « declarativeness ». For neuron networks, it remains to be seen how « meta- knowledge » can be understood and implemented.