2005 SR&ED Claim

2005 Scientific Research and Experimental Development tax credit claim

T661 - Part 2 – Scientific or Technological Project Information

Step 1 - Detailed Project Description

Project Identification: code and name

Project Number  –   1
Project Name     –   Adaptron
Project Type       –   Basic Scientific Research
Subject Area      –   Artificial Intelligence, Artificial Life

A. Scientific or technological objectives

This scientific research aims to simulate human learning and thinking using an artificial neural network (ANN - see reference) for pattern recognition with an integrated behaviour network (see reference) for action selection. The resulting agent is called Adaptron. The objective of this research is to devise an ANN that dynamically grows the number of nodes as new experiences are acquired and that prunes the nodes (forgets them) as new learning replaces old. The research aims to extend the ANN by connecting up recognition events to action sequences.

The goal of Adaptron is for it to be general purpose. This means it should be able to learn to function in any environment that can produce quantized stimuli and that obeys a set of deterministic rules whenever actions are performed within it.

Adaptron must begin with the ability to recognize simultaneously a primitive / non-reducible set of stimuli from several senses and the ability to produce a predefined set of primitive actions in parallel on several response devices. The objective is for it to first learn using novelty as a goal and by avoiding boredom. It must also be preprogrammed to recognize a subset of its stimuli as rewarding and another disjoint subset of its stimuli as punishing. With only these predetermined parameters Adaptron should learn to recognize combinations of the primitive stimuli from its environment and to perform primitive actions and combinations of primitive actions so as to minimize punishing stimuli and maximize rewarding stimuli.

Thus Adaptron must be able to “live” in an artificial environment in which it can sense the stimuli and produce actions. The environment must be 100% deterministic such that in all initial environmental states any action performed by Adaptron will always result in the same final states. All detectable dimensions of the environment must be 100% discrete – there are no continuously measurable quantities. The environment cannot change unless Adaptron performs an action i.e. there are no other agents in the environment changing its state. The environment must produce rewarding and punishing stimuli. Designing Adaptron to live in a continuous, changing and noisy environment are goals for subsequent research projects.

When the research has proven that the theories are correct and the software design is viable, Adaptron Inc. plans to promote the software for imbedding in robots and control systems.

B. Technology or knowledge base or level

Existing ANNs are built from a fixed number of nodes and the weights on the connections between these nodes are adjusted as they are trained to recognize a set of input stimuli. Devising a self-organizing ANN that also grows i.e. adds nodes as it encounters new stimuli, as a means for learning has not been attempted. Such an ANN could be used in an adaptive control system without having to reprogram it with a new set of nodes.

Existing behaviour / action networks are designed to be general-purpose networks with learning rules imbedded by the developer. They have not been integrated with ANNs nor constrained by their topology and learning algorithms. Successful integration of behavioural networks with ANNs that grow should result in an adaptive system that can build ever increasingly complicated hierarchical behaviour networks.

The area of robotics is a prime candidate for advancement through the use of Adaptron. Artificial Intelligence research into robots has been progressing for several decades and many successes have been accomplished. Many robots have been developed to perform specific tasks in very narrow environments and include some limited learning ability. However they cannot handle general-purpose situations or if they can (e.g. Brook’s robot Cog (see reference)) they do not have the ability to combine learnt behaviour into more complicated behaviour, i.e. they do not scale well. The scientific advancement that Adaptron aims to accomplish is general purpose learning and thinking software that can be imbedded in a robot such that it can learn all its knowledge from and operate in the environment in which it is placed.

C. Scientific or technological advancement

An ANN that grows hierarchically as it learns to recognize more complicated patterns of stimuli has not been invented. It is uncertain as to how the growth should be controlled. It is also unknown if any node in an ANN can be used to trigger actions. With current fixed node ANNs the actions would only be associated with final recognition nodes, not hidden nodes. With a growing ANN the actions will end up associated with hidden nodes. Strategies for using novelty, familiarity, punishment and reward, as feedback in guiding the growth of the ANN must also be discovered.

Of even more scientific uncertainty is how thinking can be introduced into the ANN. Based on the idea that thinking is a stream of expectations which effectively model experienced stimuli in a goal directed fashion various processes based on signaling between the nodes need to be invented and tested.

The fields of science that this project is involved with are Artificial Intelligence and Artificial Life. It also uses as input results obtained from the field of Cognitive Science. More specifically the areas within Artificial Intelligence are Artificial Neural Networks (ANNs) – unsupervised learning in dynamic neural networks and dynamic hierarchical Behaviour Networks.

D. Description of work in the tax year

Determination of Adaptron’s success at learning and thinking is based on the observation of its actions in test environments and by inspection of its internal memory traces and processes. The tasks performed this year were:

  1. Solve a problem when recognizing repeating series of stimuli.
  2. Find the best algorithm to deal with the relative importance of recency versus interest level for stimuli.
  3. Incorporate the idea of a level of concentration that must be overcome to attract attention and processing of subconscious recognition habits.
  4. Automate test cases for regression testing and add a more realistic front-end environment (a maze) for demonstration and testing action strategies.
  5. Try several strategies for using good and bad stimuli in learning actions.
  6. Handle stimuli from several senses simultaneously.
  7. Begin incorporating Generalization and Discrimination learning.
  8. Rework the recognition of boring stimuli and the resulting reflexive responses.

Testing of Adaptron was done in artificial environments simulated in software.

E. Supporting Information

Research notes in new Notebook started 9th Jan, 2004:

  • Idea for using Adaptron via the Internet rather than imbedding it,
  • Various relative levels of interest schemes for stimuli to attract attention,
  • Various designs for allocating good and bad weightings to habits,
  • Ideas for subconscious recognition and attracting attention,
  • Further ideas for recognition of simultaneous stimuli from several senses,

Versions of the Adaptron software that were developed are named:

Works6 thru Works9 – 33 Versions in total
Learn1 thru Learn9 – 67 Versions in total

80 Screen captures of memory dumps of test runs of Adaptron have been kept.

A logbook is kept of the daily experiments performed and the time spent doing research by the specified employee.

References:

Artificial Neural Networks:
   https://en.wikipedia.org/wiki/Artificial_neural_network

Behaviour Networks
   https://www.cse.unr.edu/~monica/Research/Control/ctrl.html
   [ This is a comprehensive alternative paper: https://www.cs.bath.ac.uk/~jjb/ftp/mphil.pdf ]

Cog Project
   http://www.ai.mit.edu/projects/humanoid-robotics-group/cog/cog.html