Evolution of Signaling in Artificial Agents

Posted 24 February 2007 by Richard B. Hoppe

Some time ago I wrote about the evolution of novel strategies for cooperation in computer models of evolutionary processes involving artificial agents with very rudimentary sensory-motor capabilities. Now another such study has appeared showing evolution of the communication of meaningful signals among artificial agents. I was in the process of writing a PT post on it when I was beaten to it by Carl Zimmer. So I'll only say that starting from scratch (random neural nets), robots who could sense their environments and move and emit light themselves, evolved in a 'field' in which there was a food source and a poison source, both of which also emitted light. Under those conditions the robots evolved to signal either the location of the food or the location of the poison. Especially in populations composed of 'kin' -- genetically related robots -- the evolution of signaling resulted in substantially more efficient food gathering and poison avoidance. Zimmer's post is here and the original paper is here. Read and enjoy. RBH

85 Comments

paul flocken · 25 February 2007

Taking this from the paper:

In all experiments, we started with completely naive robots (i.e., with randomly generated genomes that corresponded to randomly wired neural controllers) with no information about how to move and identify the food and poison sources.

Isn't this the same as saying that there was zero information at start? As in ZERO COMPLEX SPECIFIED INFORMATION? Isn't that what random strings of data have according to Dembski?; zero CSI. Then where, oh where did the information in the robot genomes come from? Sincerely, Paul

realpc · 25 February 2007

Fitness may increase, but I doubt there will be any increases in complexity or information. Depending on how you define complexity and information, of course.

Genetic algorthms have been around a long time, as arguments for NDE. Now, many decades later, they are still primitive.

I am not saying it means nothing. Just hold off on congratulating yourselves on knowing all about evolution.

caerbannog · 25 February 2007

Fitness may increase, but I doubt there will be any increases in complexity or information. Depending on how you define complexity and information, of course.

So you are basically acknowledging that evolution can take place without increases in complexity or information. Depending on how you define complexity and information, of course.

Glad you cleared things up.

Sir_Toejam · 25 February 2007

Just hold off on congratulating yourselves on knowing all about evolution.

says the one who congratulates himself on his total lack of knowledge about anything.

Sir_Toejam · 25 February 2007

in case anybody is wondering where this realpc troll came from, just do a google search on "RealPC evolution" to see where he usually posts and some of the "gems" he's put up.

hilarious.

stevaroni · 25 February 2007

Depending on how you define complexity and information, of course.

Please, be my guest. Wild Bill Dembski doesn't seem to be able to do it, maybe you can give it a shot.

Genetic algorthms have been around a long time, as arguments for NDE. Now, many decades later, they are still primitive.

Yes, but by any objective measurement they do manage to make complex information out of nothing without outside intervention, now don't they? "Dawkins Weasel" programs, for instance, annoyingly manage to create computer programs that write Shakesperian quotes all by themselves.

I am not saying it means nothing. Just hold off on congratulating yourselves on knowing all about evolution.

No, we congratulate ourselves on finding one more piece to the puzzle. A process we call "science". Another puzzle piece that any and all are free to use, even those in the ID camp, though they never seem to.

PvM · 25 February 2007

By any measure, information and complexity increased, however IDers are quick to point out that 'they don't know' and likely won't tell.

Why is it that ID revels in ignorance time after time when confronted with real science.

Sir_Toejam · 25 February 2007

Why is it that ID revels in ignorance time after time when confronted with real science.

simple. one: it's easier. two: it helps them maintain their delusions of reality just that much longer.

Henry J · 25 February 2007

Re "Under those conditions the robots evolved to signal either the location of the food or the location of the poison."

"Danger, Will Robinson!"

Henry

realpc2 · 26 February 2007

While a great number of things are explained by reductive theory (eg. galaxies, solar systems, etc), biology is somewhat novel. Is there a reductive theory for the natural evolution of intelligence? Is it a fluke, or is there a guiding being that led to this particular outcome?

Evolution might be acceptable, but without guidance, would it lead to the forms we have now? Is the general notion of complexity -- especially that of complex forms -- yet addressable by science. For a materialist there is a lot of research potential; for a spiritualist there is a lot of gaps for god.

I have no agenda, but I do think that all the realpc's out there are tipping over the same problem: how does simplistic in/organic chemistry result in an average animal that could write what I have written?

KL · 26 February 2007

realpc, you might stay on previous threads and answer questions posed to you before moving to a new thread. Folks might get the impression that you are dodging those questions.

djlactin · 26 February 2007

how does simplistic in/organic chemistry result in an average animal that could write what I have written?

1) the standard "argument from incredulity". 2) the standard sound of moving goalposts. 1) the "argument from incredulity". here's my response, as i post to all IDists: the IDist looks at a complex phenomenon, fails to grok how it came to be and gives up: "Goddidit". simple. not! consider the universe. it's large and complex beyond all imagining. how did it come to be? (grok failure: Goddidit). The fundamental problem is that the hypothetical creator must be larger and more complex than the entire universe and everything in it, especially if it is capable of monitoring all in the universe and punishing/rewarding all privileged creatures in it. I say again: larger and more complex than the entire universe. The ID/creationist 'simple solution' (cop-out from intellectual effort) requies a deity that is far more complex than the universe! How is this a solution of any kind? surely if you are uncomfortable with the idea that we cannot adequately explain how the universe came into existence, you should also be uncomfortable about postulating something even more complex as an 'explanation'. oh and what is the creator of the creator? (i'll stop the infinite loop here.) 2) the standard sound of moving goalposts. "yeah, ok, so what about consciousness mr. smarty-pants?!" all i can say is "sad."

Flint · 26 February 2007

Evolution might be acceptable, but without guidance, would it lead to the forms we have now?

This is not a question science can possibly answer, because it lies outside the boundaries that define science. It's not possible to determine whether any guiding intelligence exists using the scientific method. It's not possible to rerun biological history to see what happens. The most useful models say that evolution's details are entirely contingent - just the right (or at least workable) thing happening at the right place at the right time. Could have been anything within a wide range. Compare this, very roughly, to betting in a casino. The detailed outcomes are contingent; at any given time, anyone might win or lose. The overall odds are known; the house will win a known percentage in the long run. But on each spin, each roll, each deal, the outcome is not predictable. But this does not prevent the losers from blaming bad luck and the winners from thanking "intelligent guidance."

paul flocken · 26 February 2007

RealPC2: While a great number of things are explained by reductive theory (eg. galaxies, solar systems, etc), biology is somewhat novel.

Can you back up this assertian?

RealPC2: Is there a reductive theory for the natural evolution of intelligence?

I can't help you with one of those, but how much effort have YOU put into studying emergent properties of complex systems? Is science really so ignorant as you have painted it? Do you have a non-reductive theory of intelligence, other than goddidit, that scientists can use to interpret and investigate intelligence?

RealPC2: Is it a fluke, or is there a guiding being that led to this particular outcome? Evolution might be acceptable, but without guidance, would it lead to the forms we have now?

The form it has taken (I guess you mean Us) was a fluke. If you reran evolution from LUCA ~4gya, intelligence may or may not come about; but if it did it would most certainly not look like it does today, though it would not necessarily look radically different, either. It was a contingent process, not a deterministic one.

RealPC2: Is the general notion of complexity --- especially that of complex forms --- yet addressable by science.

What do you even mean by this? There is an entire science devoted to complexity and I am not refering to the claptrap of WAD. For a basic beginning try wikipedia and look up complexity or maybe systems theory. Many brilliant scientists have been thinking about this subject for a very long time.(And again, I am not refering to WAD.) Try learning something about the subject before you assume science knows nothing.

RealPC2: For a materialist there is a lot of research potential; for a spiritualist there is a lot of gaps for god.

OK. This is true, but it is also trivial. Furthermore, what happens as the research scientists learn more and more and the gaps continue to shrink more and more?

RealPC2: I have no agenda, but I do think that all the realpc's out there are tipping over the same problem: how does simplistic in/organic chemistry result in an average animal that could write what I have written?

Have you ever sat in an organic chemistry class? The last thing the students think is that it is simplistic, trust me. What ever gave you the idea that chemistry was simple or only capable of simple things. But setting aside this simplicity/complexity thing, do you really have some super secret, dark, insider knowledge that says it is impossible to make intelligence from organic chemisty. A lot of people would like you to spill the beans as soon as possible. Sincerely, Paul

realpc · 26 February 2007

"Dawkins Weasel" programs, for instance, annoyingly manage to create computer programs that write Shakesperian quotes all by themselves.

What exactly is that? Are you citing a computer trick as evidence for NDE?

Flint · 26 February 2007

Are you citing a computer trick as evidence for NDE?

He is citing an example of the capabilities of an algorithm. Evolution is an algorithm. The capabilities of algorithms are common in important respects. If you wish to dismiss as a "trick" everything algorithms commonly do that you find uncongenial with your requirements, fine. Nature has played a trick on us all.

GuyeFaux · 26 February 2007

What exactly is that? Are you citing a computer trick as evidence for NDE?

Yes. Specifically:

Yes, but by any objective measurement they do manage to make complex information out of nothing without outside intervention, now don't they? "Dawkins Weasel" programs, for instance, annoyingly manage to create computer programs that write Shakesperian quotes all by themselves.

GuyeFaux · 26 February 2007

And now there are three threads where Realpc has failed to back-up his assertions and answer his critics.

PvM · 26 February 2007

What exactly is that? Are you citing a computer trick as evidence for NDE?

As opposed to what? Your ignorance of science?

Raging Bee · 26 February 2007

Depending on how you define complexity and information, of course.

Well, that seems a rather crucial thing to leave out of an argument based on "information theory," doesn't it?

Do you have a definition of these terms? If so, you have yet to describe it here, or stand by it. If not, you have no argument.

Just because the "conservation of energy" theory has repeatedly proven "true," does not mean you can simply replace "energy" with "information" and still have a true theory.

djlactin · 26 February 2007

And now there are three threads where Realpc has failed to back-up his assertions and answer his critics.

Everybody needs a hobby!

PvM · 26 February 2007

Just because the "conservation of energy" theory has repeatedly proven "true," does not mean you can simply replace "energy" with "information" and still have a true theory.

On the other hand there is persistent and convincing empirical data that there exists, in parts of our space-time continuum a law of conservation of ignorance.

MarkP · 26 February 2007

Realpc said: Are you citing a computer trick as evidence for NDE?

Yes, they are a direct refutation of the claim that intelligent results cannot be gotten out of nonintelligent agents. I now brace myself for yor rendition of the Magic Pixie Dust Theory of intelligence, whereby intelligence is mysteriously transferred from people to the objects they manipulate.

David vun Kannon · 26 February 2007

Picking nits on Stevaroni's comment 162812 -

Weasel programs write strings, not programs. In this sense, they are GA, not GP.

Saying GA/GP successfully create complex information "out of nothing without outside information" is an invitation to a front loading argument.

What makes this experiment a much stronger argument is that the factor of co-evolution is explicit. The typical GA example (of the Weasel type) does not make the phenotypes of the population part of the fitness function.

GuyeFaux · 26 February 2007

The typical GA example (of the Weasel type) does not make the phenotypes of the population part of the fitness function.

Your nit-pick is in general good. Though I must say that the purpose of the Weasel program is to prove that evolution (NS+RM) can increase information, which was Stevaroni's point. The front-loading arguments would therefore be an instance of a moving goalpost.

realpc · 26 February 2007

The program is a vivid demonstration that the preservation of small changes in an evolving string of characters (or genes) can produce meaningful combinations
(Dawkins)

No, Dawkins' program did not produce meaningful combinations. The original string was "random," or meaningless, and the final string just happened to mean something within our culture.

It could have gone from
METHINKS IT IS LIKE A WEASEL
to
WDLMNLT DTJBKWIRZREZLMQCO P
with exactly the same kind of process. The program doesn't know the difference.

Yes, I understand Dawkins' point -- that shuffling plus selection works much faster than shuffling alone. Well of course, what else would we expect?

But Dawkins, probably without realizing it, makes a great unjustified leap by saying the program has produced something meaningful.

If there were no target string, and the program spontaneously generated "Hey Dawkins, you're my creator!" for example, then I would have to acknowledge the accidental generation of meaning.

Carl Rennie · 26 February 2007

But Dawkins, probably without realizing it, makes a great unjustified leap by saying the program has produced something meaningful. If there were no target string, and the program spontaneously generated "Hey Dawkins, you're my creator!" for example, then I would have to acknowledge the accidental generation of meaning.
— realpc

You're assuming our existence is meaningful. Essentially, you're taking what actually happened (evolved consciousness capable of stupid disagreements like this) and saying that it was meant to be this way, that it was inevitable that it be this way, that we were designed to be this way. Well, no, it's not, or at least it isn't provably so. The idea that the world we experience is the goal of anything is unproven. The world is what it is because of the processes that shaped it that way. Any meaning attached to it comes from the agents within it capable of assigning meaning, such as us.

MarkP · 26 February 2007

Realpc said: If there were no target string, and the program spontaneously generated "Hey Dawkins, you're my creator!" for example, then I would have to acknowledge the accidental generation of meaning.

The point of the many EA's is that nonintelligent entities can produce intelligent results, such as solving problems that in some cases the programmers themselves didn't know the answer to (ie Steiner problems). This runs contrary to the dogma of IDer/creationists like you, who miss the entire point of the exercise by focusing in on Dawkins' designation of a desired result. In most EA's that arise in these discussions, that is not the case.

Sir_Toejam · 26 February 2007

amazing.

is there any thread on this board where RealPC hasn't managed to show his utter lack of knowledge of the subject matter?

AFAICT, he's batting 1000.

anyone who can point to where anything he has said actualy exhibits even basic level understanding of the material?

really, am I missing something, or is this person just about as pure a troll as one can get?

stevaroni · 26 February 2007

David vun Kannon justifiably picks my nits thusly...

Picking nits on Stevaroni's comment 162812 - Weasel programs write strings, not programs. In this sense, they are GA, not GP. Saying GA/GP successfully create complex information "out of nothing without outside information" is an invitation to a front loading argument.

Yes, you are correct. I misspoke (mistyped?) in my haste. Dawkin's Weasel programs are instructive because they illustrate the just how vast the leap in optimization speed is when you go from sheer random chance to random chance amplified by selection. We're talking 10 to the bazillions versus a few tens of thousand iterations. Dawkin's Weasel programs illustrate that a little positive feedback easily reduces the number of generations to get from A to B to realistic numbers. But they don't really model the process of evolution. For that, you need Dawkin's Weasel generators, which is what I meant to reference in the first place. For RealPC, These are self-creating software programs that exactly mimic the actions of DNA evolution. The grand-daddy of these programs was originally described in a research paper written in 1995, Genetic Evolution of Machine Language Software - Ronald L. Crepeau NCCOSC RDTE Division San Diego, CA 92152-5000 July 1995. you can find it at h t t p :// w w w.ron-crepeau.com/index/GEMS_Article. d o c (sorry, I have to break it up like that, for some reason I can't post links) I don't know how much you know about low-level computer languages (assembly language), but it's jaw dropping. Assume, for the moment, a simple microcontroller, which runs a set of assembly language instructions. Assembly language instructions are simple things, one byte wide, and each does exactly one discrete, specific action, typically moving data around from register to register, or doing simple bit-wise math. A typical processor has about 60 instructions, the rest of the bit combinations often do nothing (no-ops). Completely determinate. In many ways just like genes, small, discrete things that do just one little step, meaningless and insignificant on it's own. These instructions live in a "chromosome" a few hundred instructions long. The chromosome is seeded with a completely random block of instructions. Essentially gibberish. A few hundred of these organisms "live" in their own block of memory. In any given generation, each has it's small block of code run, and the organisms are then compared to a random criteria to see which was "fittest". The fittest is kept, duplicated a few hundred times, and each of those offspring has a randomly selected gene randomly mutated, and the loop is run again. The selection criteria in the originally program was how closely a block of 11 bytes at one end of the memory space resembled the phrase "Hello World", but it could have been (and has since been duplicated with) any and all manner of other random criteria. Let's be clear, this is a program that writes itself and converges on a randomly chosen target without any intervention at all other than the outside environment killing off those organisms that fit worst. Just like mama nature does. Truly stunning evidence. We had a long conversation last summer about all this, check it out, it's really frickin' amazing. h t t p :// w w w.pandasthumb.org/archives/2006/06/evolution_of_co_2.html

realpc · 27 February 2007

nonintelligent entities can produce intelligent results, such as solving problems that in some cases the programmers themselves didn't know the answer

Of course a program can give me an answer I didn't know -- there would be no reason to write a program if it couldn't!
But I have to give it the algorithms, or algorithms to generate the algoritms, etc.

stevaroni,
So Dawkins improved his little program by creating an algorithm-generator. And you are madly impressed.

I don't know how much you know about low-level computer languages (assembly language), but it's jaw dropping.

What is "jaw dropping" about low-level computer languages? What does that have to do with evolution?

Flint · 27 February 2007

realpc: You need to read what people tell you. Otherwise, you are doomed to repeating the same corrected errors in a infinite loop!

But I have to give it the algorithms, or algorithms to generate the algoritms, etc.

The point of what was explained to you was, these various levels of algorithm were not known. It was up to the evolutionary process to invent the algorithms. All that was known was that whatever "worked" (i.e. didn't crash) was preserved, and became the parent of the next generation. Let run a few tens of thousands of iterations, and solutions no engineer would ever have dreamed of emerge. So do solutions similar to existing solutions but a lot more efficient. The idea behind all this is, evolution is a simple algorithm which has been running for a few billion years, and which in the process has ramified into all the life we see around us. Even the algorithm of evolution has ITSELF evolved over this time. No Cosmic Engineers required - indeed, they'd have surely made things worse.

What is "jaw dropping" about low-level computer languages? What does that have to do with evolution?

Again, you need to read what people are trying to tell you. What is "jaw dropping" is that the processor has only a very few basic instructions, each of which does only a very tiny and simple operation. Just like genes. Yet left to evolve, these few simple instructions can organize themselves into routines of many instructions, and the routines can organize themselves into amazingly complex and creative programs. And so the creationist-on-the-street would look at these programs and apply denial-by-incredulity - "I simply cannot believe that a stone simple process involving a small set of stone simple instructions could have evolved this big complex program all by itself! Some Higher Power must have written it." Even those who set the feedback process loose have to pinch themselves to make sure they're not dreaming. Positive feedback is an incredibly powerful concept. What Dawkins is doing is finding simple ways to illustrate this power. And that power is indeed impressive, to anyone willing to watch it work and understand what it means.

realpc · 27 February 2007

the routines can organize themselves into amazingly complex and creative programs.

No, nothing described here has been amazingly complex. And I studied artificial intelligence, and never heard of anything convincing. Designers are always required, ultimately, and machine intelligence is always very limited.

I think this is yet one more example of people seeing what they prefer in ambigous data. AI has always been used as an important argument for scientific materialism. Look at all the disappointing dead ends AI has traveled.

I think AI has turned out to be one more reminder of human limitations and our lack of real understanding. Our technology is great and, at least to us, amazing. But our understanding of nature is minimal. The more we learn about it, the more confusing it all becomes. Look at string theory.

Flint · 27 February 2007

realpc:

No, nothing described here has been amazingly complex.

OK, whatever. Those who understand what's happening and how are amazed. You are clearly committed to denial.

And I studied artificial intelligence, and never heard of anything convincing.

Who said anything about AI? We're talking about evolving programs in this case, not about AI. You're entirely correct, if the goal is to evolve something that imitates human consciousness, this goal has not been met. But this is not what we're talking about.

I think this is yet one more example of people seeing what they prefer in ambigous data.

I think you have made this point, however inadvertently. You have taken clear data, and deemed it ambiguous not because it is, but because you require this. As for what the data ARE, you see nothing because you are committed to seeing nothing. I can picture you watching Galileo dropping two different sized rocks, seeing them fall together, and ("knowing" the larger one MUST fall faster) deciding the data are ambiguous and those who see them fall at the same speed are "seeing what they prefer." I see nothing inherently wrong with religious faith. But faith threatened by clear evidence is misplaced. God isn't trying to trick you and is not lying to you. Evolution clearly happens. Wouldn't it be better to have faith in how God really works, rather than deny God's works because you can't understand them? Do you think God is impressed that you defend ignorance in His name?

Raging Bee · 27 February 2007

But our understanding of nature is minimal. The more we learn about it, the more confusing it all becomes.

Speak for yourself, pal. Just because you can't handle the complex reality, doesn't mean no one else can either. Also, if you don't understand "nature" yourself, you're not in any position to judge anyone else's understanding of it.

Look at string theory.

Some of us already are. Yes, it's mind-boggling, but one reason for that is that it's a very new theory, and has a long way to go before scientists even agree on what it is, exactly, let alone how to test it. Most new theories begin that way.

KL · 27 February 2007

And NO ONE is asking that we teach string theory in high school.

GuyeFaux · 27 February 2007

And I studied artificial intelligence, and never heard of anything convincing.

I thought you studied psychology. Oh, I get it: like most psych students, you went into the discipline for the purpose of self-analysis. So you really do know a bunch about what being artificially intelligent is. Anyhow, realpc's anti-GA arguments are pretty standard creationist non-sense. Even the weasel algorithm is an explicit but trivial example of GA's producing new information, refuting at least one of his ramblings. Anything else is just shifting the goalpost. Pretty sure that next he'll say that GA's need to be programmed by human beings. Oh wait, he already did:

But I have to give it the algorithms, or algorithms to generate the algoritms, etc.

GuyeFaux · 27 February 2007

AI has always been used as an important argument for scientific materialism. Look at all the disappointing dead ends AI has traveled.

And this is more or less bullshit; particularly coming from a somebody who's claimed to have studied both AI and psychology. Here's a clue: try playing your computer in chess. The AI winter sucked, but I invite you to point out anything in AI which is a dead-end. You should have no problems, having studied AI.

realpc · 27 February 2007

Evolution clearly happens.

Ok, this makes about the billionth time I have said it here -- I BELIEVE IN EVOLUTION.
A couple more times:
I BELIEVE IN EVOLUTION.
I BELIEVE IN EVOLUTION.

try playing your computer in chess.

There are lots of things my computer does better than I can. That is why I have a computer. But it only does work that can be automated (broken down into explicit algorithms and programmed). It has no autonomous intelligence.

A chess program follows the rules it was given. It can out-perform humans because it does certain kinds of operations much faster than we can. Not because it's smarter.

Like any machine, a computer is something we design for a purpose. Machines are superior to us in many ways, but they are still our creations and they have no independent existence or consciousness (and they never will).

And by the way I studied experimental psychology as a branch of cognitive science. I'm a computer scientist, not a shrink.

GuyeFaux · 27 February 2007

A chess program follows the rules it was given. It can out-perform humans because it does certain kinds of operations much faster than we can. Not because it's smarter.

Wrong. Unless you're a GM, nowadays the computer is actually way more likely to know more about chess than you. A good example of this is the program Hiarchs, which uses a knowledge base of chess rules. And if you don't like chess (where raw computation helps a lot), how about TD-gammon which, in its original incarnation, used a neural net to outplay most people. In particular, TD-gammon falsifies this:

But it only does work that can be automated (broken down into explicit algorithms and programmed).

since TD-gammon's learning algorithm was completely unassisted, i.e. it learned by playing itself. So humans were not responsible for any so-called automation. Feel free to now shift the goal-post to saying that human beings designed the learning algorithm.

...they have no independent existence or consciousness (and they never will).

Yet another un-scientific assertion.

GuyeFaux · 27 February 2007

Designers are always required, ultimately, and machine intelligence is always very limited.

As has been amply demonstrated, designers are not always required. No designer other than RM+NS was required to produce a meaningful program in stevaroni's example. Also, no designer was required to produce meaningful behavior in the paper referenced in this post. Claiming that designers were required for the actual software and the neural nets is certainly missing the point. You should have no problem seeing why; connectionists in cognitive science use the same types of models.

realpc · 27 February 2007

connectionists in cognitive science use the same types of models.

Oh yes, connectionism was supposed to be the answer to how the brain works. AI researchers are worried soon the machines will be smarter than we are, and become our masters. How long have they been worrying about that?

I am not losing sleep over that scenario.

GuyeFaux · 27 February 2007

Oh yes, connectionism was supposed to be the answer to how the brain works.... not about to lose sleep ... robots ... blah blah

What the hell does this have to do with the applicability of models to the real world? If you don't like connectionism, substitute your favorite model instead, and you can see how absurd it is to complain about human-designed models.

Flint · 27 February 2007

Oh yes, connectionism was supposed to be the answer to how the brain works. AI researchers are worried soon the machines will be smarter than we are, and become our masters. How long have they been worrying about that? I am not losing sleep over that scenario.

Few if any of us are. So what? The point was that algorithms can be creative. When this was pointed out, you tried to change the subect to talk about computational speed, rather than creativity. When corrected, you tried to change the subject and talk about how a higher intelligence is required to write the algorithms. When corrected, you tried to change the subject and talk about artificial intelligence goals. When corrected, you tried to change the subject and talk about science fiction scenarios. So once again, let's try to focus on the subject: algorithms entailing positive feedback mechanisms (such as selection) can be creative, without any "higher intelligence" being involved at all. Your turn.

realpc · 27 February 2007

algorithms entailing positive feedback mechanisms (such as selection) can be creative

I don't know how you define "creative" and I don't want to get tangled in that debate. But certainly, hardly anyone would describe the Dawkins trick as an example of creativity. Or any similar genetic programs. The goal is programmed, and guidance toward the goal is, if not programmed, at least laid out pretty clearly.

As I said, we do not have to start worrying, at least not yet, about these things getting out of control and taking over the world.

And by the way, I am not kidding. Some well-known person in the computer industry expressed that concern very recently. Oh yeah, it was Bill Joy (thank you google).

GuyeFaux · 27 February 2007

I don't know how you define "creative" and I don't want to get tangled in that debate.

That is the debate, and certainly what ID claims to be about. Ample evidence has been presented to you where GA's have come up with creative solutions to problems. You're arguing that they were not in fact creative.

But certainly, hardly anyone would describe the Dawkins trick as an example of creativity.

Nobody said that. They're examples, at least, of information gain, which you claimed couldn't happen.

Or any similar genetic programs. The goal is programmed, and guidance toward the goal is, if not programmed, at least laid out pretty clearly.

Ahh, now the old "information is sneaked in via the fitness function" argument. It's a bad argument; it's like saying that the solution to a traveling salesperson problem is encoded in its statement.

As I said, we do not have to start worrying, at least not yet, about these things getting out of control and taking over the world. And by the way, I am not kidding. Some well-known person in the computer industry expressed that concern very recently. Oh yeah, it was Bill Joy (thank you google).

Which has nothing to do with anything, but of course you're welcome to try to shift the goalpost and contribute to Flint's running list.

David vun Kannon · 27 February 2007

At the risk of running my own little sidebar conversation with stevaroni...

I wouldn't call Weasel generators a good example of the process of evolution. Again, their fitness function doesn't include the (phenotypes of) the other members of the population.

On the Crepeau paper, and the long ramble last year that you cite, the paper isn't a major contribution to the "peer reviewed literature". (3 citations within 1 year, by one author) The author conflates several claims about the use of large, generic instruction sets in GP with claims about the use of memory in GP. He then undercuts his own argument that generic instruction sets are useful by biasing the instruction set in favor of the test problem. There is only one test problem, which makes drawing a conclusion about his thesis difficult, at best.

Futher, it wasn't quite apropos of the original question posed by GilDodgen, a point not well appreciated in the subsequent discussion.

It's a pity, because there is a possibility of doing real science is comparing the difficulty of evolving a "Hello, World" generator in machine language with the difficulty of evolving a "Hello, World" generator in C. Crepeau gives us one data point (for Z80 machine language). GilD throws down the gauntlet, but doesn't do the experiment himself. Neither do any evo proponents, they just assume its possible based on Crepeau's data point.

Perhaps I'm giving GilDodgen too much credit, but to me it would be interesting to know that linear strings of machine instructions take effort X, regular expressions take effort Y, and BNF specified programs take effort Z, to evolve. You could then make predictions like "It is unlikely that DNA has certain features (statement boundaries, block structure, function invocation and return) because these features are very hard to evolve."

ID proponents sometimes yammer about DNA being a language, but if it is any more complex than

S = (G|A|T|C)*

I haven't read about it.

Flint · 27 February 2007

The goal is programmed, and guidance toward the goal is, if not programmed, at least laid out pretty clearly.

This statement needs minor correction. 1) The goal is not programmed. The GA algorithms define their own goal, different every run, and changing all the time. 2) Guidance toward anything at all is NOT programmed, which is exactly the point. If it were programmed, the illustration would be useless. 3) So nothing is laid out at all, much less "pretty clearly". We start with a few simple rules: replicate with variation, inherit variations, and compete for resources. That's all evolution works with, so that's all the programs work with. Same result: entirely creative, contingent, unpredictable paths to unexpected places. Other than that, you're right!

realpc · 27 February 2007

They're [the Dawkins programs] examples, at least, of information gain, which you claimed couldn't happen.

What information is gained? Random shuffling of characters, guided by artificial selection towards a pre-determined goal.

It is hard to agree on a definition of "information," though. We generally mean something that "informs" us, tells us some news, makes us know something we did not know before.

Biological evolution has shown a great increase in information. New forms and functions grow out of previously-existing forms and functions. The same with human cultural evolution.

GuyeFaux · 27 February 2007

BNF specified programs take effort Z, to evolve.

I don't know if it answers your question, but BNF specified programs (say, in lambda calculus) have certainly been evolved. For instance, your question prompted me to read this cool paper: Evolution of lambda-expressions through Genetic Programming

Flint · 27 February 2007

Biological evolution has shown a great increase in information. New forms and functions grow out of previously-existing forms and functions. The same with human cultural evolution.

And the same with the GA programs, yes. Positive feedback functions based on the Three Rules (variation, inheritance, and competition) produce an increase in information, however you choose to measure it. Economies follow similar principles, with similar results. Nobody need to "manage" any of these processes, and nobody does (those who have tried to manage economies only distorted the normal processes, to their own detriment).

MarkP · 27 February 2007

Realpc exposed himself thusly: Of course a program can give me an answer I didn't know --- there would be no reason to write a program if it couldn't!

Surely you do not mean this. Computers rarely give answers the programmer doesn't know. We write the programs because the computer can do what we do faster and more accurately. Until EAs that is...

But certainly, hardly anyone would describe the Dawkins trick as an example of creativity. Or any similar genetic programs. The goal is programmed, and guidance toward the goal is, if not programmed, at least laid out pretty clearly.

You are unaware of the facts. The goals and the guidance towards it are not programmed into EAs, any more than me tossing you a Rubik's cube and telling you to get every side the same enables you to solve it. GAs are allowed to randomly vary, just like genes in nature do, and the weak are culled, just like nature does, and viola, we get complicated solutions to problems, exactly as the IDers/creationists claimed we wouldn't. And rather than accept that their ideology is in need of repair, they wave their hands, make baseless assertions about front-loading, and generally ignore reality. There was a devastating demonstration of this posted here by Dave Thomas with Steiner problems (shortest total length of straight lines connecting a set of points). Someone better with links than I am can direct you there. The EAs produced solutions unknown to the programmers, and NOT contained in the code.

What information is gained? Random shuffling of characters, guided by artificial selection towards a pre-determined goal.

Again, the goal is not predetermined in these exercises. That's the fact you guys just can't seem to bring yourselves to digest. Furthermore, if the production of a solution that the programmers were unaware of is not an increase in information, what the hell is it?

GuyeFaux · 27 February 2007

What information is gained? Random shuffling of characters, guided by artificial selection towards a pre-determined goal.

Yes; that roughly characterizes Shannon information, and refutes this:

Fitness may increase, but I doubt there will be any increases in complexity or information. Depending on how you define complexity and information, of course.

if by information you meant loss of randomness. If you meant something else, provide a formal definition. The point was not that the WEASEL program did anything interesting; the point was that it was a direct test of your assertion, and your assertion was therefore wrong. You can rebut this by providingng a formal definition of information which is not tested/refuted by the WEASEL experiment.

It is hard to agree on a definition of "information," though. We generally mean something that "informs" us, tells us some news, makes us know something we did not know before.

Give a precise definition, and you might start to make a point.

Biological evolution has shown a great increase in information. New forms and functions grow out of previously-existing forms and functions.

Ditto...?

The same with human cultural evolution.

Ditto...? Once again, it's your turn. What are you talking about when you say "information"? Be precise, otherwise you will have done what William "Divine Wind" Dembski has been doing for a living: using a lot of words to say nothing.

MarkP · 27 February 2007

Flint wrote: Economies follow similar principles, with similar results. Nobody need to "manage" any of these processes, and nobody does (those who have tried to manage economies only distorted the normal processes, to their own detriment).

Interesting. Economics provides a perfect counter-example to the claim that intelligence is more capable of managing great complexities than are mindless processes, and it is one the Republican-dominated IDer/creationists would othewise trumpet: Free economies outperform managed ones. Yet free economies are, in every way that matters, mindless. Managed economies are the ones that have direct intellectual influence, and they prove not up to the task. So the basic ID conclusion is backwards. When we see something that is responsive to its surroundings, and complex beyond our ability to understand or manage, that is not an indication that an intelligence is behind it. It is not consistent with what we know of intelligent sources. Complexity implies mindlessness, not intelligence. For example, what are the more complicated houses, those that were designed in total by an intelligence, or those that had no intellectual guidance and were basically put together a little at a time? It is no contest.

realpc · 27 February 2007

What are you talking about when you say "information"?

It can be defined in different ways, none of them perfect.

A message contains no information if it is entirely predictable, or if it's entirely unpredictable (random). So one way to define information is as a message where the probably is somewhere between 1 and 0.

But this can be interpreted in various ways. The probability of a coin landing heads is between 1 and 0, but the probability of landing heads OR tails is 1.

Sometimes the set of possibilities is defined, and sometimes it isn't. When I turn on the news I can predict it to some extent -- I know there will be something about Anna Nicole and Britney -- but there is always a chance of something surprising.

The surprises generated by computer programs are, as far as I know, more like coin-tossing than like news. There is a set of expected possibilities -- we know with certainty the result will be from that set, we just don't know which one it will be.

We have never seen human-like creativity in a computer program. They never make jokes, for example.

And life-like programs have been around for half a century. So it doesn't seem like a breakthrough is around the corner.

MarkP · 27 February 2007

realpc dissembled thusly: The surprises generated by computer programs are, as far as I know, more like coin-tossing than like news. There is a set of expected possibilities --- we know with certainty the result will be from that set, we just don't know which one it will be.

You are simply ignoring reality. EAs produce answers that are NOT, repeat NOT, part of any known expected set of outcomes. THAT IS THE WHOLE FRIGGIN POINT!!!

It is hard to agree on a definition of "information," though. We generally mean something that "informs" us, tells us some news, makes us know something we did not know before.

Then EAs produce information by your own definition. The EAs set to the Steiner problem produced answers unknown by the programmers, and outperformed the mere humans who tried to find solutions. They informed us.

We have never seen human-like creativity in a computer program. They never make jokes, for example.

Have you missed the news stories of computers that paint and write? I saw them years ago, and no doubt the ones now are better. Deep Blue beat Kasparov in part because it was able to play strategically, not just tactically. You need to stop assuming that because YOU don't know of something, or because YOU have never seen something, that it doesn't exist, because frankly you don't seem to get out much.

GuyeFaux · 27 February 2007

What are you talking about when you say "information"?

It can be defined in different ways, none of them perfect. And you've given none of them. Try again; I'm not going to guess what you were trying to say in the rest of the post.

We have never seen human-like creativity in a computer program.

Focus! What's information? Creativity? Give a definition that actually means something.

GuyeFaux · 27 February 2007

And if by "creativity" you mean a novel approach to solve problems, there are many examples. The paper I referenced above, "Evolution of lambda-expressions through Genetic Programming", described an experiment that solves a problem Church himself could not solve. Also, the solutions are completely novel.

MarkP · 27 February 2007

Here's a great article about some evolutionary algorithms in robots:

http://scienceblogs.com/loom/2007/02/24/evolving_robotspeak.php

The robots, in the total absence of any programming telling them to do so, not only developed languages, but developed different languages. They are certainly simple languages, granted, but the fact that they created them at all is the death knell of all this "design implies a designer", "unintelligence cannot produce intelligence", "the target is frontloaded" nonsense. It's been refuted, empirically, in the only way creationists have said they would accept: right in front of their primate faces.

It's reality. Deal.

stevaroni · 27 February 2007

RealPC wrote...

So Dawkins improved his little program by creating an algorithm-generator. And you are madly impressed. and... Designers are always required,

No, that's the whole point of the thing, Crepeau's program wrote itself. Designers are not always requited, nor is intelligence. That's big fish, regardless of your efforts to idly dismiss it. And I studied artificial intelligence, and never heard of anything convincing. Study harder. And start with the Crepeau paper.

stevaroni · 27 February 2007
RealPC wrote...
So Dawkins improved his little program by creating an algorithm-generator. And you are madly impressed.
and...
Designers are always required,
No, that's the whole point of the thing, Crepeau's program wrote itself. Designers are not always requited, nor is intelligence. That's big fish, regardless of your chafingly Dembski-esque efforts to idly dismiss it. And I studied artificial intelligence, and never heard of anything convincing. Study harder. And start with the Crepeau paper.

realpc · 28 February 2007

What's information? Creativity? Give a definition that actually means something.
They are philosophical concepts that cannot be defined precisely. Everyone thinks the meanings are obvious, but they cannot state them logically.
There are things we can do, many others we cannot. Giving precise logical definitions of words like creativity, information, love, evolution, for example, is beyond our ability.

Flint · 28 February 2007
Giving precise logical definitions of words like creativity, information, love, evolution, for example, is beyond our ability.
But of course this is not true. We can easily give precise logical definitions of all these terms. What is beyond our ability is to get people to agree with any definition that offends their preconceptions. "Information" has been given several different precise logical definitions, each definition serving a somewhat different purpose. But we must recognize that unless we provide a definition and specify that this is the definition we are using, we aren't going to communicate clearly. The position you take, of throwing up your hands and denying any possibility of defining these terms, while still insisting that evolution is not creative and adds no information, is not honest. You could not make this claim without some notion of what these concepts mean; you won't pin down the meaning you are using because doing so makes the goalposts so much more difficult to move.

GuyeFaux · 28 February 2007
They are philosophical concepts that cannot be defined precisely.
Wrong, twice. Ever heard of information theory?
Everyone thinks the meanings are obvious, but they cannot state them logically.
Wrong, twice. Speak for yourself; The meanings are not obvious, but they can be stated logically. In terms of your informal usages of the term, which is in general not objectionable unless you start making assertions which hinge on its definition, I think MarkP's points are apt on how it applies to GAs.

realpc · 28 February 2007

while still insisting that evolution is not creative and adds no information
I believe that evolution is creative and adds information. I think I have stated this repeatedly.

realpc · 28 February 2007

Ever heard of information theory?
We can define information within a restricted context. But when talking about things like biological or cultural evolution, the context is unrestricted.

Glen Davidson · 28 February 2007
We can define information within a restricted context. But when talking about things like biological or cultural evolution, the context is unrestricted.
Okay, obviously you do know nothing about information theory. Information is well-defined physically. Hint: the most universal definition relates to entropy, as in, each increase in entropy is an increase in information. And when we're talking about biological evolution the context is as restricted as in any other physical system. Subsets of information are important as well, however most information increases during biological evolution that we care about are quite well understood. In cultural evolution, yes, the relevant information would be difficult to define. I would also say that love and creativity cannot be given precise definitions, if by that we assume that we mean "precese and meaningful" definitions. True, I can define love as grains of sand quite precisely, but I can't define love precisely in a meaningful sense. Information and evolution, on the other hand, are far more amenable to definition. Glen D http://tinyurl.com/35s39o

GuyeFaux · 28 February 2007
I believe that evolution is creative and adds information. I think I have stated this repeatedly.
— realpc
Actually, no you haven't. What you did is insinuate that the algorithm of evolution (as demonstrated by GAs) cannot innovate. E.g., stuff you've said:
Fitness may increase, but I doubt there will be any increases in complexity or information.

Evolution might be acceptable, but without guidance, would it lead to the forms we have now? Is the general notion of complexity --- especially that of complex forms --- yet addressable by science.

the routines can organize themselves into amazingly complex and creative programs.
No, nothing described here has been amazingly complex. And I studied artificial intelligence, and never heard of anything convincing. Designers are always required, ultimately, and machine intelligence is always very limited.
They're [the Dawkins programs] examples, at least, of information gain, which you claimed couldn't happen.
What information is gained? I'm willing to give the benefit of the doubt here (the quotes above are without context), so go on and explain the contradiction. In what meaningful sense is evolution creative in which GAs are not creative?

MarkP · 28 February 2007
Realpc said: A message contains no information if it is entirely predictable, or if it's entirely unpredictable (random). So one way to define information is as a message where the probably is somewhere between 1 and 0. But this can be interpreted in various ways. The probability of a coin landing heads is between 1 and 0, but the probability of landing heads OR tails is 1.
Should we really be taking seriously someone who could write the above? "Probably" [sic] of what?! The rest is similarly confused. It sounds like something written by a 10-year-old, copying random phrases from an introductory probability text he clearly doesn't understand. Obviously this guy is making it up as he goes.

realpc · 28 February 2007
As information increases, entropy decreases. But you have to decide what you mean by entropy.
If high entropy means low predictability (high randomness), then high information would mean high predictabilty. Which of course it doesn't.
In information theory, the quantity of information carried by a message increases as the set of possible states selected from by the message increases.
A coin toss has only two possible outcomes. If the coin evolves so that it has three sides instead of two, then maybe we can say the complexity of the system has increased.

realpc · 28 February 2007
But measuring complexity really has to be subjective. If I compare a fruit fly to a zebra, for example, I feel that the zebra is much more complex. But how do you quantify that?

realpc · 28 February 2007
Ignoring the insults, what I am trying to express is the idea (maybe hard to grasp for someone as literal-minded as MarkP) that it can be hard to quantify information in complex contexts. Human language and culture, or biological evolution, for example.
You are claiming simple computer programs can lead to an increase in information, and assuming we can apply the same measures to real life. How do we measure the complexity of a species, so as to compare it to others?

Raging Bee · 28 February 2007
...what I am trying to express is the idea...that it can be hard to quantify information in complex contexts.
You've expressed this idea quite well, and repeatedly, thank you. And we're quite aware of this fact. And we've been trying to express to you that your argument is completely invalid unless, and until, you manage to define and quantify "information." Bleating about how "hard" that is doesn't reinforce any of your arguments; it only makes you sound like one of those talking Barbie dolls.
Just let us know when you manage it, okay? Then -- and only then -- we can do the math and see whether this argument of yours has any substance. Until then, it's nothing but brown air.

MarkP · 28 February 2007
Science is as literal-minded as it gets. Want to be flowery and vague, the poetry room is down the hall.

Glen Davidson · 28 February 2007
As information increases, entropy decreases. But you have to decide what you mean by entropy.
OK, the only thing you need is an education, and to quit blithering on about things you understand not at all, moron. Where'd you learn this BS, from the IDists, the creationists, or from some other pig-ignorant pseudo-scientists? Anyway, it's obvious that you're incapable of an intelligent discussion of any of these matters. Glen D http://tinyurl.com/35s

realpc · 28 February 2007
Science is as literal-minded as it gets
MarkP, Progress in science depends on intuition and philosophical speculation. The ID controversy hinges on definitions of information and complexity, which is a philosophical problem. Once terms have been defined and a problem has been clearly stated, data can be collected and analysed. Literal-mindedness is just not enough. I am skeptical that we humans will ever be able to completely define the system of which we are components. In a way, that means I'm an agnostic. (However, I can still have opinions based on what limited data is available, including personal experience.) I would rather not call anti-Darwinism Intelligent Design, actually, because ID does not sound agnostic. It sounds like it's claiming to know more than it could possibly know. I would prefer to say that Darwinism (or neo-Darwinism, whatever you prefer) has not been demonstrated, and can never be demonstrated. It is neither falsifiable nor verifiable. The Darwinist claim is that ONLY chance and selection are required to create complex information, and that GAs create complex information using only chance and selection. Proving that the origin of new species does not require any other causal factors. I do not think that settles the question. Whatever is created by GAs is on a completely different level from DNA, for one thing. It's a difference of quality, not just quantity.

Raging Bee · 28 February 2007
Progress in science depends on intuition and philosophical speculation.
Progress in science consists ENTIRELY of, and is measured by, literal, physical, measurable, repeatable results. "Intuition and philosophical speculation" are worthwhile only insofar as they lead to such results. No literal results, no progress.
...In a way, that means I'm an agnostic.
I thought you said you were Catholic. Of course, since you completely ignore the Catholic Church's well-reasoned position on evolution, and science in general, that previous statement was probably meaningless.
I would prefer to say that Darwinism (or neo-Darwinism, whatever you prefer) has not been demonstrated, and can never be demonstrated. It is neither falsifiable nor verifiable.
What you "prefer to say" is irrelevent. Your statement is just plain false, and the fact that you make it proves that you have no clue what you are talking about. Stop pretending that pseudo-mystical know-nothingism is an enlightened position, and go back to bed.

GuyeFaux · 28 February 2007
Progress in science depends on intuition and philosophical speculation.
What institute awarded you with a PhD for thinking like this?
The ID controversy hinges on definitions of information and complexity, which is a philosophical problem.
No, the ID controversy hinges on its proponents doing anything of scientific worth. Incidentally, insofar as the "theory" of ID is built on top of irreducible complexity, ID does hinge on definitions of information and complexity. If the ID movement was interested in doing science, it would provide formal definitions for these terms and others so that it could make testable predictions; but of course, it's not, and they haven't. Instead, ID proponents flail about using terms that have no meaning, which allows them to move the goalpost as soon as their ideas are refuted. Much like you, realpc, as you have flailed around with the term "information".

Sir_Toejam · 28 February 2007
What institute awarded you with a PhD for thinking like this?
the imaginary one that awarded him his parapsychology degree. I've got it! RPC is really Dr. Peter Venkman! bust any ghosts lately, doc?

realpc · 28 February 2007

Progress in science consists ENTIRELY of, and is measured by, literal, physical, measurable, repeatable results. "Intuition and philosophical speculation" are worthwhile only insofar as they lead to such results. No literal results, no progress.

Yes, there have to be measurable results, to answer the questions. But there would be no questions without philosophical speculation.

...In a way, that means I'm an agnostic. I thought you said you were Catholic.

A perfect example of stereotype-induced hallucination! In your stereotype, anyone who not a neo-Darwinist must be a Catholic! (or right-wing Christian fundamentalist). I am not even Christian. Hope that puts a dent in your stereotype.

Sir_Toejam · 28 February 2007
But there would be no questions without philosophical speculation.
but of course! Ray Stanz and Egon Spengler never would have created the ghost-trap unless there was a philosophic debate as to the existence of ghosts to begin with! RPC working on his grad stundent thesis: "All right, I'm gonna turn over the next card. Concentrate... I want you to tell me what you think it is."

MarkP · 28 February 2007
Realpc, Master of the Obvious, opined thusly: ...there would be no questions without philosophical speculation
There would be no questions without speculation period, regardless of what sort it is. Speculation with undefined terms is not philosophical speculation, it is mere noise. As long as the terms "complexity" and "information" are undefined, they can be replaced with "bla bla bla" with no loss of meaning. That applies to almost every post you've made.
I would rather not call anti-Darwinism Intelligent Design, actually, because ID does not sound agnostic. It sounds like it's claiming to know more than it could possibly know.
It doesn't just sound that way, it is that way, since the IDers have never demonstrated any way to distinguish actual design from apparent design. It is far from agnostic. Congratulations, you finally got something right, and in the process helped demonstrate that random shuffling of words will eventually produce something of value.

Sir_Toejam · 28 February 2007
Congratulations, you finally got something right, and in the process helped demonstrate that random shuffling of words will eventually produce something of value.
like a broken clock, that even still can be essentially correct twice per day.

Flint · 28 February 2007
the IDers have never demonstrated any way to distinguish actual design from apparent design.
On the contrary, they have indeed. They demonstrate it constantly. They use the traditional Religious Method - they SAY so! They point to one thing and say it's actual, and point to something else and say it's apparent, and thus so it is. And when two IDers disagree about whether something is actual or only apparent, they resolve this conflict using another part of the Religious Method - they have a schism, whereby each side SAYS they possess Truth, and the other side wallows in Error. Your blind spot is, you're looking for a distinction based on the merits of the object. But this is looking in the wrong direction. Objects exist in reality, but design exists in the imagination, in the perception. This is where distinctions must be drawn.

MarkP · 28 February 2007
I stand corrected.