About the Meta-evolver

Author:  Mitchell Timin
Posted:  06/11/2006

Driftwood :: Harnessing the Power of Many Computers for Simulated Evolution

Author:   Mitchell Timin
Posted:  03/21/2004

(Note to reader: some degree of familiarity with simulated evolution is assumed, whether it be GA, GP, or any other variant of evolutionary computation.)

Introduction - Simulated evolution, for non-trivial problems, requires large amounts of CPU time. We propose to enlist the power of many computers in order to greatly reduce the calendar time involved. Another, equally important, objective, is to overcome the tendency of a single computer's simulated population to lose all of its diversity, hence stagnating at a less than optimal solution. Our plan is to simulate what happens in nature when there are isolated populations of a single species, but there is occasional migration between them. A classical example is a group of islands, where the land animals form isolated populations, except than some individuals, perhaps lacking territories of their own, will hop onto pieces of driftwood, and be carried to a different island. Hence our proposed software is a system to simulate the migration of individuals between islands.

The proposed system consists of a pair of programs that run concurrently in each of many computers. Call these two programs the "Evolver" and the "Migrator". The Evolver performs simulated evolution in some way, but in addition it periodically reads and writes individual population members from/to disk files. The Migrator also reads and writes these files. The ones that it reads, it sends to other computers in the system. The ones that it writes are first received as messages from other computers in the system. Both the Migrator and the Evolver remove files from the disk after reading them. The Evolver, when it writes a file, also removes the corresponding member from its internal population. When it receives a file, that "immigrant" will replace the most recent "emigrant". The Migrator sends each file to only one other computer, selecting the recipient at random. Hence the total number of population members does not grow nor shrink as a result of these operations, nor is any cloning performed.

The Migrator will maintain a list of all IP number of the machines currently comprising the system. The operators of the various computers will each know some of these IPs, so that there will be an initial partial list when Migrator begins execution. After that, the various copies of Migrator will share their lists so that all machines will have all IPs. When a machine is taken offline, that information will also be shared, so that all can remove that IP. Migrator is an example of a peer-to-peer system, in that there is not a single central server. Instead, a server runs in every machine that is part of the driftwood network.

The programs known as 4Play and EvSail are examples of Evolvers. (http://sourceforge.net/projects/annevolve) Here are some details of their operation: When running, they write a chromosome to disk about every 5 to 10 seconds. These files all have a name like chrom###.ann, where ### may be anything from 000 through 999. These 3-digit numbers are picked at random. The resulting files are binary, roughly 5K in size. They also remove files with names like that from the disk, at the same rate. They won't remove the most recent file that was written. These files are removed from the disk, but they appear in the population internally. Similarly, the ones written to the disk are removed from the internal population.

Migrator, running concurrently with Evolver, also removes such files from the disk; it should be able to grab them before Evolver can, in most cases. Those files that it removes it sends to another copy of itself on another machine. It only sends the file to one other machine. Of course it also receives files that are sent by other copies of itself; these it writes to disk, where Evolver will probably read them in.

A program to do this now exists. It's in Python, written by Rex Sheasby. It's architecture is somewhat different than described above, but the end result is the same. Rex's Migrator has both a client and a server portion. These run concurrently, and each has a text display window. The server casts the driftwood pieces adrift, i.e., it removes chromosome files from the disk and sends them to other computers. The client brings the driftwood ashore, i.e., it receives messages containing chromosome images and write them as disk files. Another feature of this approach is that the server only sends files that are requested by clients. File transfer is initiated by a client request message to the server. The server then attempts to comply with the request. This program runs under both linux and Windows, and requires a single open port. It is presently being tested and perfected, but it works: chromosome files are periodically and automatically collected from all the known servers, and requested files are sent out.

So imagine there are a few dozen machines, in different locations, each running Evolver and Migrator. These are ordinary PCs, running linux or Windows. Migrator is sending files between machines, in all possible paths. The files originate within Evolver which writes them to disk. The local copy of Migrator removes each one and sends it somewhere. The files which are received are written to disk, where the local Evolver copy sucks them into its population.

It is worth noting again that nodes may be added or removed at any time. This means that once some computers are engaged in this "Driftwood" network, it can then grow, and grow, and grow........

Meet ANNEvolve's founder and leader

Author:  Mitchell Timin
Posted:  02/16/2004

4-Play Procedure Analysis

Author:  Mr. Emile Richard
Posted:  01/23/2004

Shakespeare, Darwin, and the Monkeys

Author:  Mitchell Timin
Posted:  12/26/2003

How Simulated Evolution Works

Author:  Mitchell Timin
Posted:  11/16/2003

Meet Annevolve's skydiving, mouseball collecting Unix Admin

Author:  Eric Anderson
Posted:  11/14/2003

Species Learning and a Hypothesis About Brain Learning

Author:  Mitchell Timin
Posted:  10/02/2003

When doing GA, expect a very large variance in the time required to accomplish a certain amount of evolution.

Author:  Kent Pault Dolan
Posted:  09/09/2003

An Aspect of Natural Evolution

Author:  Mitchell Timin
Posted:  08/31/2003

Genuine Artificial Intelligence  :)

Author:  Mitchell Timin
Posted:  08/27/2003