IDSIA Postdoc, iBonsai, PhD Prelim, Sapience Engine, etc.

It's been several months since I posted here, mainly because I've been busy working on my research code and my PhD preliminary proposal/presentation. Here's a quick summary of things since March...

IDSIA
In March I applied for a postdoc position at Juergen Schmidhuber's lab at IDSIA in Lugano Switzerland. At the AGI 2009 conference I met with Juergen to interview for this. Shortly afterwards I was offered and accepted the position, starting in January 2010! So now I need to hurry and wrap up the PhD before then, which is stressful (because I hate trying to write up work until it's truly ready), but also good (because I'm gonna go crazy if I'm in school much longer). The postdoc will involve research related to artificial curiosity, which is very exciting to me, especially considering that I'll be working with Juergen, one of the most important researchers in the field. I'll be working with the iCub robot (pictured below) at IDSIA for much of the time. (Next month I'll be attending the RobotCub summer school to learn more about the iCub.)

iBonsai

iBonsai (details here) was released for the iPhone/iPod touch on December 28, 2008. My primary motivation for making this was to make money so I can continue my AI research unconstrained. (Despite the fact that I'm in it for the money, I can't stand to put out a sub-par app... I did spend several months of nights and weekends tweaking the algorithm and visual style, and I'm proud of the result.)

After 6 months of sales I can't say I'm ready to retire off the sales, but I'd say it has done better than expected. The picture below shows the top 100 apps list in the iTunes App Store on 6-9-09. Note that on this day iBonsai is right behind Myst and is beating several popular apps like Fieldrunners, Koi Pond, Ocarina, and SimCity. This high rank is mainly due to being featured on the App Store during that week (in Staff Picks in the US, What's Hot in several other countries). Overall iBonsai has been downloaded 95,000 times (mostly free downloads during short promotional periods to gain exposure).

I would really like to make some more apps, though I don't know when I'll find time to do it. I made a list of possible app ideas last fall and decided to start with something small (iBonsai) just to test the waters. Maybe when I finish the postdoc I'll continue iPhone app development as a source of income. Or maybe I'll find time on the weekends. We'll see. I must say it is much more satisfying to be able to point to a real product and say, "Wanna see my iPhone app?" instead of, "Wanna see this project I've been working on? I posted the source code on my website. You have a C++ compiler, right?"

PhD Preliminary Proposal

I wrote up a research proposal for my PhD in April and May, then presented it to my committee in early June. The title of the proposal (and tentative title of the final thesis) is "Sapience: A Brain Inspired Cognitive Architecture." The proposal is mainly a high-level description of my Sapience Architecture, which represents the bulk of the conceptual work I've been doing over the past several years. This architecture is fully implementable; each component can (and has been) implemented in software and tested in simulated or robotic bodies. It is comprised of 5 main components inspired by the brain's sensorimotor cortex, hippocampus, basal ganglia, cerebellum, and prefrontal cortex. And it's intended to be useful, with behavioral shaping provided by programmer-defined reinforcements.

It's been a little while since I've written up any research, so it was a bit of a struggle to convert my ideas into such a formal document. I feel like I'm good at writing, given enough time (and admittedly, this proposal could have used more time...); it's just very mentally taxing to produce a document worth other people's attention. One thing that helped was to create diagrams. I made a high-level architecture diagram (shown below) and a more detailed one for each of the main components. I think these really help clarify the design much better than words alone.

I'm glad to have written this proposal because it forced me to articulate several of my underlying assumptions. For example, I describe the "purpose" of this architecture in terms of reinforcement learning; the system's objectives include external reinforcements (i.e. programmer-defined goals) which makes the system practically useful, and internal curiosity reinforcements which provides a source of autonomous self-development even without a human teacher. Corresponding to these rewards are two useful metrics for measuring learning progress: external reward intake (goal achievement rate) and world model improvement rate (information gain).

At this point I have a software implementation of each of the 5 components, along with the necessary code to integrate them into a cohesive "software brain" (described next). Now the primary work involves testing each component in isolation and in combination with the other components. Part of the testing process will involve a simulated human arm, complete with proprioceptive, tactile, and vision sensors and servo motors (shown below).

Sapience Engine

For my PhD thesis I'm calling the conceptual work the Sapience Architecture, which is basically a blueprint for a practical, engineered thinking system. The corresponding software implementation of this architecture is called the Sapience Engine. I plan to use this software as a research platform for my postdoc position at IDSIA, hopefully applying it to the iCub.

I'm pretty excited about this software. When I started grad school in 2003, I originally wanted to build a black box software brain, and that's exactly what the Sapience Engine is. It's shaping up to be a pretty powerful system, scalable to high-dimensional inputs and outputs, with a very general behavioral shaping system. The API is very simple. It's written in C++ and has Python bindings (via ctypes). It has built-in multithreading and runtime performance profiling. It has been structured in a way that can be easily adapted for clustered computers (e.g., via MPI). Several pieces could even be GPU-parallelized.

I also have written a very useful real-time probe tool, designed as a client/server pair: I can link any Sapience Engine-powered program with the Sapience probe server, which provides access to various internal variables, then run a separate Sapience probe client (earlier version pictured below) which displays real-time plots, 2D array visualizations, neural network adaptation, etc. This "brain debugger" is turning out to be a crucial tool in the testing process.

I Don't Want to Do Academic Research

Ok, this statement doesn't really make sense considering that I'm currently finishing a PhD and preparing to start a postdoc position. I guess what I mean is that the idea of being a traditional academic researcher does not appeal to me. I hate the idea of structuring my research in order to generate publications at a fixed rate. That just doesn't make sense to me. (Well, it makes sense in general in that it makes the academic system scalable to lots of researchers, but I don't think it's the best strategy for every project or person.) Also, I think my medium of choice is software development, not publications. I would rather have a list of open source software projects on my website than a list of papers. A well-designed, well-written, timely software release can have an overall impact similar to that of an influential paper.

I would prefer to work on big ideas, to work on them long enough to see them succeed (even if that means several years without publishing), then write up papers after the fact. Some ideas just seem to work better that way. Lots of trial and error and engineering work up front, then scientific analysis later. For example, the automobile, the world wide web, the large hadron collider... It doesn't always make sense to do a lot of scientific work until the engineering work is complete. And the same might be true for general AI.

On a personal level, I think what bothers me most about traditional academic research is the expectations. I work best when I feel unconstrained by external pressures (to make money, to publish papers, to reach certain artificial objectives, etc.). In that sense, I suppose what I'm looking for long-term is complete freedom of expression within the realm of AI research. I expect my work to be very useful, but I just don't want to feel constrained. I want to be an engineer with the motivation of an artist. Over the next several years I intend to find a way to make that happen, ideally by making a big chunk of money up front to support several years of unconstrained creativity.