IDSIA Postdoc, iBonsai, PhD Prelim, Sapience Engine, etc.

It's been several months since I posted here, mainly because I've been busy working on my research code and my PhD preliminary proposal/presentation. Here's a quick summary of things since March...

IDSIA
In March I applied for a postdoc position at Juergen Schmidhuber's lab at IDSIA in Lugano Switzerland. At the AGI 2009 conference I met with Juergen to interview for this. Shortly afterwards I was offered and accepted the position, starting in January 2010! So now I need to hurry and wrap up the PhD before then, which is stressful (because I hate trying to write up work until it's truly ready), but also good (because I'm gonna go crazy if I'm in school much longer). The postdoc will involve research related to artificial curiosity, which is very exciting to me, especially considering that I'll be working with Juergen, one of the most important researchers in the field. I'll be working with the iCub robot (pictured below) at IDSIA for much of the time. (Next month I'll be attending the RobotCub summer school to learn more about the iCub.)

iBonsai

iBonsai (details here) was released for the iPhone/iPod touch on December 28, 2008. My primary motivation for making this was to make money so I can continue my AI research unconstrained. (Despite the fact that I'm in it for the money, I can't stand to put out a sub-par app... I did spend several months of nights and weekends tweaking the algorithm and visual style, and I'm proud of the result.)

After 6 months of sales I can't say I'm ready to retire off the sales, but I'd say it has done better than expected. The picture below shows the top 100 apps list in the iTunes App Store on 6-9-09. Note that on this day iBonsai is right behind Myst and is beating several popular apps like Fieldrunners, Koi Pond, Ocarina, and SimCity. This high rank is mainly due to being featured on the App Store during that week (in Staff Picks in the US, What's Hot in several other countries). Overall iBonsai has been downloaded 95,000 times (mostly free downloads during short promotional periods to gain exposure).

I would really like to make some more apps, though I don't know when I'll find time to do it. I made a list of possible app ideas last fall and decided to start with something small (iBonsai) just to test the waters. Maybe when I finish the postdoc I'll continue iPhone app development as a source of income. Or maybe I'll find time on the weekends. We'll see. I must say it is much more satisfying to be able to point to a real product and say, "Wanna see my iPhone app?" instead of, "Wanna see this project I've been working on? I posted the source code on my website. You have a C++ compiler, right?"

PhD Preliminary Proposal

I wrote up a research proposal for my PhD in April and May, then presented it to my committee in early June. The title of the proposal (and tentative title of the final thesis) is "Sapience: A Brain Inspired Cognitive Architecture." The proposal is mainly a high-level description of my Sapience Architecture, which represents the bulk of the conceptual work I've been doing over the past several years. This architecture is fully implementable; each component can (and has been) implemented in software and tested in simulated or robotic bodies. It is comprised of 5 main components inspired by the brain's sensorimotor cortex, hippocampus, basal ganglia, cerebellum, and prefrontal cortex. And it's intended to be useful, with behavioral shaping provided by programmer-defined reinforcements.

It's been a little while since I've written up any research, so it was a bit of a struggle to convert my ideas into such a formal document. I feel like I'm good at writing, given enough time (and admittedly, this proposal could have used more time...); it's just very mentally taxing to produce a document worth other people's attention. One thing that helped was to create diagrams. I made a high-level architecture diagram (shown below) and a more detailed one for each of the main components. I think these really help clarify the design much better than words alone.

I'm glad to have written this proposal because it forced me to articulate several of my underlying assumptions. For example, I describe the "purpose" of this architecture in terms of reinforcement learning; the system's objectives include external reinforcements (i.e. programmer-defined goals) which makes the system practically useful, and internal curiosity reinforcements which provides a source of autonomous self-development even without a human teacher. Corresponding to these rewards are two useful metrics for measuring learning progress: external reward intake (goal achievement rate) and world model improvement rate (information gain).

At this point I have a software implementation of each of the 5 components, along with the necessary code to integrate them into a cohesive "software brain" (described next). Now the primary work involves testing each component in isolation and in combination with the other components. Part of the testing process will involve a simulated human arm, complete with proprioceptive, tactile, and vision sensors and servo motors (shown below).

Sapience Engine

For my PhD thesis I'm calling the conceptual work the Sapience Architecture, which is basically a blueprint for a practical, engineered thinking system. The corresponding software implementation of this architecture is called the Sapience Engine. I plan to use this software as a research platform for my postdoc position at IDSIA, hopefully applying it to the iCub.

I'm pretty excited about this software. When I started grad school in 2003, I originally wanted to build a black box software brain, and that's exactly what the Sapience Engine is. It's shaping up to be a pretty powerful system, scalable to high-dimensional inputs and outputs, with a very general behavioral shaping system. The API is very simple. It's written in C++ and has Python bindings (via ctypes). It has built-in multithreading and runtime performance profiling. It has been structured in a way that can be easily adapted for clustered computers (e.g., via MPI). Several pieces could even be GPU-parallelized.

I also have written a very useful real-time probe tool, designed as a client/server pair: I can link any Sapience Engine-powered program with the Sapience probe server, which provides access to various internal variables, then run a separate Sapience probe client (earlier version pictured below) which displays real-time plots, 2D array visualizations, neural network adaptation, etc. This "brain debugger" is turning out to be a crucial tool in the testing process.

I Don't Want to Do Academic Research

Ok, this statement doesn't really make sense considering that I'm currently finishing a PhD and preparing to start a postdoc position. I guess what I mean is that the idea of being a traditional academic researcher does not appeal to me. I hate the idea of structuring my research in order to generate publications at a fixed rate. That just doesn't make sense to me. (Well, it makes sense in general in that it makes the academic system scalable to lots of researchers, but I don't think it's the best strategy for every project or person.) Also, I think my medium of choice is software development, not publications. I would rather have a list of open source software projects on my website than a list of papers. A well-designed, well-written, timely software release can have an overall impact similar to that of an influential paper.

I would prefer to work on big ideas, to work on them long enough to see them succeed (even if that means several years without publishing), then write up papers after the fact. Some ideas just seem to work better that way. Lots of trial and error and engineering work up front, then scientific analysis later. For example, the automobile, the world wide web, the large hadron collider... It doesn't always make sense to do a lot of scientific work until the engineering work is complete. And the same might be true for general AI.

On a personal level, I think what bothers me most about traditional academic research is the expectations. I work best when I feel unconstrained by external pressures (to make money, to publish papers, to reach certain artificial objectives, etc.). In that sense, I suppose what I'm looking for long-term is complete freedom of expression within the realm of AI research. I expect my work to be very useful, but I just don't want to feel constrained. I want to be an engineer with the motivation of an artist. Over the next several years I intend to find a way to make that happen, ideally by making a big chunk of money up front to support several years of unconstrained creativity.

AGI 2009 Conference

I attended the 2nd AGI conference a few weeks back. The goal of the conference is to help organize the field of artificial general intelligence (AGI).


Juergen Schmidhuber (pictured above) gave the keynote, entitled "The New AI." I was especially interested in the part of his talk on artificial curiosity. Marcus Hutter gave two good paper presentations, "Feature Markov Decision Processes" and "Feature Dynamic Bayesian Networks." John Laird's talk on the SOAR architecture included a helpful definition of cognitive architectures (specific set of fixed mechanisms) vs. frameworks (neural nets, rule-based systems, etc.). There was an interesting mixture of people there (maybe 100 total?) from academia and AGI-based startup companies and organizations.

In order for the AGI field to move forward in a cohesive, organized way, it will be important to define standard evaluation metrics. (The conference session on creating an "AGI preschool" discussed this issue.) This seems to be one of the biggest hurdles in the near term. Communication among researchers is already difficult since nearly everyone uses a different set of terminology, but the lack of standard evaluation and comparison methods makes it even more difficult. Producing solid metrics might even be the most crucial step here... once you know what you're measuring, it's much easier to work towards that goal. However, general intelligence is really hard to measure, even in humans. The best starting point is probably to start from our current definitions of general intelligence, which are usually in the form of "an agent must be successful at achieving goals/maximizing rewards in a wide variety of environments." So I'm thinking that a good practical approach is the one described in a paper by Thomas Hazy, Michael Frank, & Randall O'Reilly (Towards an Executive Without a Homunculus: Computational Models of the Prefrontal Cortex/Basal Ganglia System): "To the extent that the same basic model can account for a progressively wider range of data, it provides confidence that the model is capturing some critical core elements of cognitive function." So we can build a standardized, ever-growing repository of small tasks, each with a clear measure of success/failure (either binary or scalar). Then we can subject our AGI systems to the entire test set and measure general intelligence performance as the fraction of tests passed. Our confidence that the metric is useful should then be proportional to the number and variety of tasks in the test set. I can't think of a better, simpler way to measure general intelligence than this.

At the conference I showed a live demo of my sensory cortex model learning from natural images, along with the following poster (full-sized image available from my website):

Practical Mind Control

Effective mind control (for any purpose) is not about making people do things they don't want to do. It's about changing what they want. Then they think they still have free will over their own decisions.

To accomplish this, simply expose them to your idea/product/meme repeatedly. Here's how it works:

• The more we experience something, the better we can imagine it.
  
• The better we can imagine something, the more we choose to think about it.
  
• The more we think about something, the more it influences our actions.
  

The more we experience something, the better we can imagine it.

Our brains build representations of things as they receive data samples of those things; sensations physically change our mental hardware. The more samples from a particular data source (foods, music styles, visual art styles, places), the more accurate our mental representation of that source.

The better we can imagine something, the more we choose to think about it.

Our minds are attracted to some thoughts over others.  The question of which thoughts are most attractive can be answered by the theory of curiosity rewards: our brains produce internal rewards as long as they can improve at predicting new data. The better our mental representation of something, the more we are able to notice, appreciate, predict, and enjoy its complexities (for example, the complex flavor patterns in coffee, wine, chocolate, olives, and cheese). These situations can be intrinsically rewarding as long as we get better at understanding/predicting them. (When we can no longer improve, boredom ensues.)

Thus, we perform a type of unconscious mental rehearsal of a small subset of possible thoughts. Our attention is most focused on those ideas which provide the most rewarding progress towards better prediction, which tend to be the ones about which we have significant experience (and thus mental representation). As we continually gravitate towards those ideas, they become the easiest to recall.

(In some cases there might even be a positive feedback loop here: the better we can imagine something, the more it evokes curiosity rewards, the more we want to experience it, the more we do experience it, the better our mental representation, the better we can imagine it...  The process is bootstrapped by an outside force which provides the initial exposure, but this feedback loop keeps it going.)

The more we think about something, the more it influences our actions.

The burden of making a decision (e.g., which of several products to buy) is lessened by having a short list of options in mind.

If the cost of making a decision is factored in (which is usually the case), we must find a balance between picking the best option and minimizing the time needed to make the decision itself. The weighting of these two factors depends on the cost of the decision outcome vs. the cost of wasting time making the decision; we can afford to spend more time if the decision outcome is more important.

More exposure to/thinking about one option makes it easier to recall vs. others, which shortens the decision process, possibly to the point where the other options are not even worth recalling...

Conclusion

By the transitive property, we tend to choose things we have experienced most. Thus, to make people like/choose your idea/product/art, simply expose them to it repeatedly.

George Costanza utilizes this mind control technique in the Seinfeld episode The Chicken Roaster. Heather: "Alright George, I'll be honest. The first time we went out, I found you very irritating, but after seeing you for a couple of times, you sorta got stuck in my head... Co-stanza!" (to the tune of "By Mennen").

Note that this only works for emotionally/value neutral things.  If a person already has an aversion to something, simple exposure might not be enough to make it attractive. But repeated exposure to an initially neutral thing/place/idea tends to make it more attractive than other still-neutral options.

This is the essence of advertising in general, the idea that "any press is good press," and the rich-get-richer type of driving force behind all kinds of pop culture phenomena. It is practical on a personal level (self mind control) in terms of discovering new tastes. Our dislike for certain things should not be considered an intrinsic property of ourselves, but rather a set of tastes which we have not yet acquired. Expect not to like things at first; everything is an acquired taste which can be enjoyed with a certain amount of practice (although the investment might not always be worth the effort).