I've been thinking for a while about how to make the training system work smoothly with the desired behaviors. So far I've been creating a separate C++ class for each desired behavior. This works fairly well, but they usually take a while to tweak. It would be nice to have a separate script file for each desired behavior. So... I've been learning to use Lua. I think Lua scripts will work nicely for this purpose. I won't need anything too fancy, and it seems like Lua is getting pretty popular with game developers.
I hit a roadblock yesterday when I was thinking about how Lua would work with Verve. The users will have full control over their character models and virtual environments, but the desired behaviors (whether scripts or C++ functions/classes) will need to have full access to the character model and possibly the environment to decide how well the character's controller is doing.
The way I did this with my desired behavior C++ classes was this (using "walking" as an example): each frame/time step a pointer to the character would be passed to the Walking class. This class would examine the character's body (how far it had moved forward, how many steps it had taken) and update the fitness score. The problem with doing this in Lua scripts is that the users would have to create all the Lua/C++ bindings to let the script access the character and environment. This would work, but it makes the API uglier.
So I came up with this idea: the user will create a set of metrics to measure in the C++ code. For example, let's say the user is training a simulated robot to stack blocks. Each time step the user could call Robot->GetMetrics() and maybe Environment->GetMetrics(). These return the number of steps the robot has taken, how much energy he has used, the height of his head, the number of blocks stacked up in the environment, etc. Notice that some of the numbers might not be needed for all desired behaviors; they are just a bunch of number that might be used. Then, the numbers all get passed to the Stacking Blocks Lua script which measures only the number of blocks stacked and returns a fitness score. Another Lua script (Jumping) might use other numbers (the head height) to calculate a fitness score.
I'll test this method to see how well it works. I want the API to be very clean, and I don't want the users to have to write tons of extra code. To give them the freedom to use their own character code, though, they'll still have to do some extra work.
I think I may also give them the option of using Lua scripts or not. They may just want to write the desired behaviors in C++ anyway (maybe because they don't know/want to learn Lua or because the Lua scripts might slow down the training process).
Saturday, June 26, 2004
XML files
A couple of days ago I changed most of the data files to use XML. I'm using the TinyXML parser to load a save these files. Specifically, the files that are now XML files are 1) the NEAT parameters file, 2) the "controller" files (neural networks), and 3) the "Training Script" file. Here's a quick description of each one:
1.) The NEAT parameters file holds all of the genetic algorithm parameters (population size, probabilities of various mutations, etc.).
2.) A controller file holds info about the neurons and connections that make up a neural network.
3.) The Training Script file is sort of a high level description of a training sequence. The data it contains include: the location of the NEAT parameters file, whether to seed the population with a trained controller, the location of that controller file, whether to save the trained controllers, whether to save statistics, the output directory for all output files, how often to save controller files (in terms of generations), how many generations should elapse before quitting, how many runs/repetitions of the whole training sequence to perform, the number of sensors and effector the trained controller should have, and whether the desired behavior is "oscillatory." This final parameter decides whether the neural nets should initially contain a lot of recurrent connections. Certain oscillatory behaviors (e.g. walking) need a lot of these from the start.
Also, the Training Script contains Training Stages (XML elements) that each describe a different stage in the evolution process (i.e. incremental evolution). Each stage can contain one or more desired behaviors and a fitness goal. For example, to train a controller to walk, the first Training Stage might use a "Oscillate Legs" desired behavior. Once the fitness of a controller reaches the fitness goal for that stage, the training system switches to the next training stage, which could be "Maximize Forward Distance" in the case of walking. Note that each stage can have more than one desired behavior. You could use a "Minimize Energy Expended" desired behavior along with a walking behavior to make the controller learn to walk while minimizing energy used.
One file still type still needs to be switched to XML: the statistics file. This file will contain information about fitness (best and average), neural net complexity (# of neurons and connections), and speciation.
By the way, one of the reasons I chose XML is because a guy named Derek James recently wrote a NeuroEvolution Visualization Toolkit - http://nevt.sourceforge.net This takes XML files (neural nets and statistics files) and converts them to SVG (scalable vector graphics) files. This should come in handy later.
1.) The NEAT parameters file holds all of the genetic algorithm parameters (population size, probabilities of various mutations, etc.).
2.) A controller file holds info about the neurons and connections that make up a neural network.
3.) The Training Script file is sort of a high level description of a training sequence. The data it contains include: the location of the NEAT parameters file, whether to seed the population with a trained controller, the location of that controller file, whether to save the trained controllers, whether to save statistics, the output directory for all output files, how often to save controller files (in terms of generations), how many generations should elapse before quitting, how many runs/repetitions of the whole training sequence to perform, the number of sensors and effector the trained controller should have, and whether the desired behavior is "oscillatory." This final parameter decides whether the neural nets should initially contain a lot of recurrent connections. Certain oscillatory behaviors (e.g. walking) need a lot of these from the start.
Also, the Training Script contains Training Stages (XML elements) that each describe a different stage in the evolution process (i.e. incremental evolution). Each stage can contain one or more desired behaviors and a fitness goal. For example, to train a controller to walk, the first Training Stage might use a "Oscillate Legs" desired behavior. Once the fitness of a controller reaches the fitness goal for that stage, the training system switches to the next training stage, which could be "Maximize Forward Distance" in the case of walking. Note that each stage can have more than one desired behavior. You could use a "Minimize Energy Expended" desired behavior along with a walking behavior to make the controller learn to walk while minimizing energy used.
One file still type still needs to be switched to XML: the statistics file. This file will contain information about fitness (best and average), neural net complexity (# of neurons and connections), and speciation.
By the way, one of the reasons I chose XML is because a guy named Derek James recently wrote a NeuroEvolution Visualization Toolkit - http://nevt.sourceforge.net This takes XML files (neural nets and statistics files) and converts them to SVG (scalable vector graphics) files. This should come in handy later.
Friday, June 25, 2004
Old news...
Here are the posts that I made on my old Verve news site before it was replaced by this blog:
Friday, June 18, 2004
Added a Bibliography page.
Thursday, June 17, 2004
An executive summary and technical white paper have been posted in the Documentation section.
Friday, June 11, 2004
The Autonomous Virtual Humans project has been restructured. The new system is much more general in that it will work with any physics engine and any character model, not just humans. The new project has been named Verve and has a new website: www.vrac.iastate.edu/~streeter/verve/main.html.
Friday, June 18, 2004
Added a Bibliography page.
Thursday, June 17, 2004
An executive summary and technical white paper have been posted in the Documentation section.
Friday, June 11, 2004
The Autonomous Virtual Humans project has been restructured. The new system is much more general in that it will work with any physics engine and any character model, not just humans. The new project has been named Verve and has a new website: www.vrac.iastate.edu/~streeter/verve/main.html.
Verve blog is now online
I just created a new blog on blogger.com for the Verve project. I'll use this space in the future to keep track of ideas I have. Hopefully it will encourage potential Verve users to give me feedback, too. :)
Subscribe to:
Posts (Atom)