2008 Scientific Research and Experimental Development tax credit claim
T661 - Part 2 – Scientific or Technological Project Information
Step 1 - Detailed Project Description
Project Identification: code and name
Project Number – 1
Project Name – Adaptron
Project Type – Basic Scientific Research
Subject Area – Artificial Intelligence, Simulation of Adaptive Behavior, Autonomous Mental Development
A. Scientific or technological objectives
This scientific research aims to simulate human learning and thinking using an artificial neural network (ANN - see reference 1.) for pattern recognition with an integrated behavior network (see reference 2.) for action selection. The resulting agent is called Adaptron. The objective of this research is to devise an ANN that dynamically grows the number of nodes as new experiences are acquired and that prunes the nodes (forgets them) as new learning replaces old. The research aims to extend the ANN by connecting up recognition events to action sequences.
The goal of Adaptron is for it to be general purpose. This means it should be able to learn how to function in any environment that can produce quantized stimuli and that obeys a set of deterministic rules whenever actions are performed. This objective is also described by researchers at Michigan State University (see reference 5.).
Adaptron must begin with the ability to recognize simultaneously a primitive / non-reducible set of stimuli from several senses and the ability to produce a predefined set of primitive actions in parallel on several response devices. The objective is for it to first learn using novelty as a goal and by avoiding boredom. It must also be preprogrammed to recognize a subset of its stimuli as rewarding and another disjoint subset of its stimuli as punishing. With only these predetermined parameters Adaptron should learn to recognize combinations of the primitive stimuli from its environment and to perform primitive actions and combinations of primitive actions so as to explore, minimize punishing stimuli and maximize rewarding stimuli.
Thus Adaptron must be able to “live” in an artificial environment in which it can sense the stimuli and produce actions. The environment must be 100% deterministic such that in all initial environmental states any action performed by Adaptron will always result in the same final states. All detectable dimensions of the environment must be 100% discrete – there are no continuously measurable quantities. The environment cannot change unless Adaptron performs an action i.e. there are no other agents in the environment changing its state. The environment must produce rewarding and punishing stimuli. Designing Adaptron to live in a continuous, changing and noisy environment are goals for subsequent research projects.
When the research has proven that the theories are correct and the software design is viable, Adaptron Inc. plans to promote the software for imbedding in robots and control systems.
B. Technology or knowledge base or level
Existing ANNs are built from a fixed number of nodes and the weights on the connections between these nodes are adjusted as they are trained to recognize a set of input stimuli. Devising a self-organizing ANN that also grows i.e. adds nodes as it encounters new stimuli, as a means for learning has not been attempted. Such an ANN could be used in an adaptive control system without having to reprogram it with a new set of nodes.
Existing behavior / action networks are designed to be general-purpose networks with learning rules imbedded by the developer. They have not been integrated with ANNs nor constrained by their topology and learning algorithms. Successful integration of behavioral networks with ANNs that grow should result in an adaptive system that can build ever increasingly complicated hierarchical behavior networks. Growing hierarchical networks are available from Numenta Inc. (reference 6.) however they are not ANNs and don’t include thinking.
The area of robotics is a prime candidate for advancement through the use of Adaptron. Artificial Intelligence research into robots has been progressing for several decades and many successes have been accomplished. Many robots have been developed to perform specific tasks in very narrow environments and include some limited learning ability. However they cannot handle general-purpose situations or if they can (e.g. Brook’s robot Cog (see reference 3.)) they do not have the ability to combine learnt behavior into more complicated behavior, i.e. they do not scale well. Current research in this field is best followed by the International Society for Adaptive Behavior (see reference 4.). The scientific advancement that Adaptron aims to accomplish is general purpose learning and thinking software that can be imbedded in a robot such that it can learn all its knowledge from and operate in the environment in which it is placed.
C. Scientific or technological advancement
An ANN that grows hierarchically as it learns to recognize more complicated patterns of stimuli has not been invented. It is uncertain as to how the growth should be controlled. It is also unknown if any node in an ANN can be used to trigger actions. With current fixed node ANNs the actions would only be associated with final recognition nodes, not hidden nodes. With a growing ANN the actions will end up associated with hidden nodes. Strategies for using novelty, familiarity, punishment and reward, as feedback in guiding the growth of the ANN must also be discovered.
Of even more scientific uncertainty is how thinking can be introduced into the ANN. Various processes based on signaling between the nodes need to be invented and tested. These are based on the idea that thinking is a stream of expectations / predictions which effectively model experiences in a goal directed fashion in order to predict consequences of future actions.
The fields of science that this project is involved with are Artificial Intelligence, Artificial Life and Simulation of Adaptive Behavior. Results obtained from the field of Cognitive Science are also used as input. More specifically the areas within Artificial Intelligence are Artificial Neural Networks (ANNs) – unsupervised learning in dynamic neural networks and dynamic hierarchical Behavior Networks. The field is also called Autonomous Mental Development.
D. Description of work in the tax year
Determination of Adaptron’s success at learning and thinking is based on the observation of its actions in artificial test environments simulated in software and by inspection of its internal memory traces and processes. The major areas of research performed this year were:
- Changed the attention algorithm so it is interrupted only by stimuli that are interesting above the concentration level.
- Added subconscious background habit execution to recognize any unexpected stimuli.
- Implemented multi-sensor discrete stimuli from a single sense as an array of values.
- Discovered an improved hierarchical memory structure and implemented it.
- Tried to devise an algorithm to recognize graduated stimuli from a sense comprised of multiple sensors (as in hearing).
- Added an algorithm to recognize change as a stimulus.
E. Supporting Information
44 pages of research notes in the “A-Habit Learning.doc” file:
- Stimuli readings versus changes in stimuli for graduated senses.
- Generalization versus specialization / discrimination.
- Action selection for exploration.
- Alternative tree structures for storing sequences of stimuli.
- Re-investigation of objects, edges and motion on a graduated sense.
Versions of the Adaptron software that were developed are named:
Cogni1 thru Cogni9 – 80 versions in total
Grad1 thru Grad6 – 58 versions in total
Recog3 thru Recog6 – 31 versions in total
49 Screen captures of memory dumps of test runs of Adaptron have been kept.
A logbook is kept of the daily experiments performed and the time spent doing research by the specified employee.
1. Artificial Neural Networks:
http://www.dacs.dtic.mil/techs/neural/neural_ToC.html [website no longer active]
2. Behavior Networks
4. International Society for Adaptive Behavior
5. Developmental Robots – A New Paradigm by Juyang Weng and Yilu Zhang at the Michigan State University
6. Hierarchical Temporal Memory - Concepts, Theory, and Terminology by Jeff Hawkins and Dileep George, Numenta Inc.
http://www.numenta.com/Numenta_HTM_Concepts.pdf [web page no longer available]