Friday, May 18, 2007

Modified Delta Learning Rule Part 2

Another modification to my delta learning rule is motivated by my preference for working with real time values (t=0.13s, t=0.16s, t=0.19s, ...) as opposed to abstract discrete time values (t=1, t=2, t=3, ...). This distinction is important for interactive real time applications, like robotics and video games.

I want to be able to specify how much error is reduced per second (e.g., an error reduction rate of 0.3 means that the error is reduced by 30% per second). Here's how I do it:

error_reduction_rate = 1 - exp(-step_size / error_reduction_tc)

"step_size" is the amount of real elapsed time between weight updates (it's best if this is a fixed constant, both for stable learning and to avoid recomputing the error reduction rate constantly). "error_reduction_tc" is a time constant which determines how fast errors are reduced.

For example, an error reduction time constant of 2.0 means that the error will be reduced to about 37% of its original value after 2.0 seconds of updates. If weight updates occur every 0.05 seconds, this yields an error reduction rate of 0.02469. If we change the time constant to 0.1, leaving the step size at 0.05, the error reduction rate jumps up to 0.3935 since the time constant is much faster. Note: it's important to keep the step size smaller than the time constant; otherwise, things can get unstable.

7 comments:

Johnnyburn said...

Rock on, Tyler. I love it when you talk about learning rules.

Seriously!

Will Dwinnell said...

This is interesting. Have you compared your modified technique to the conventional delta rule on real data?

Tyler Streeter said...

I have compared it to the conventional delta rule on a few toy problems, and it appears to work just as well. As noted in the update on my previous post, the normalization part is actually an existing method (normalized LMS), which is good because it is probably well-tested. The use of a learning time constant doesn't appear to pose any problems on real data; it just changes the learning rate from discrete time steps to a continuous time representation.

Will Dwinnell said...

It would be interesting to see whether your method improves performance on benchmark data (such as the UCI machine learning or data mining data sets). If you perform any such comparisons, I for one would be glad to hear about it.

Tyler Streeter said...

There appears to be an existing body of literature on the normalized LMS rule and how it compares to other learning rules. here is one example. Since my research isn't really focused on learning rules, I don't plan on doing such a comparison anytime soon.

Anonymous said...

VGiL2I Please write anything else!

Anonymous said...

Good job!