1986

21st Apr. 1986

Matching the longest pattern of events allows for recognizing a change in the environment - deviation from plan just as described in "Dynamic Memory" by Schank. [This is an additional comment to the ones of 10th Feb 1985]

I need to spend some time thinking about the process of 'starting to do' - The cue, the goal idea, the desire for the goal and the doing of the associated sequence.

17th May 1986

Why doesn't a baby keep on sucking at mother's breast forever? There is a direct reward - milk for the action performed and the situation is continuously there - baby is not forcefully removed! Reason: The reward is decreased as the baby reaches satisfaction level - The value of the reward diminishes as the baby is less hungry - primary desire is fulfilled.

Along with the ability to recognize a sequence of inputs comes the primitive attention - instructions to break down the sequence i.e. get next input, get previous input. These feel like they should be short-term memory type operations.

The environment in which adaptron could operate is determined by what input and output classes it has. E.g. humans have

   Input Class           Output Class

   Ears-Sound           Voice-Speech

   Eyes-Light             None

   Feel/Nerves,

   Pressure/heat,

   Muscle/location      Muscles-motion

   Nose-Smell             None / body aromas?

   Mouth-Taste          None

Only in the Feel-Muscles class of input/output devices are their reward/punishment feedback available for a robot. Punishment may not be the best word to use because it implies outside influence by an active agent. Let us call it pain, A bump on the head from slipping on a shiny floor. Humans can get reward and pain from the mouth-taste class of devices where there is also the primitive hunger drive.

Let us devise an environment in which we want adaptron to learn- survive and adapt. With or without other active agents.

What is the goal - purpose of a human? Once his primitive drives are fulfilled to an adequate level - not hungry, no sex drive, happy, warm, comfortable, clothed, sheltered etc. the only thing left is intellectual pursuit, whether, scientific, artistic, social interchange which all boils down to new experiences. What about the desire to work? Is that also a desire for new experiences? Yes, because if you do a repetitive job its boring - no challenge - no change. What about the desire to procreate - is that an avoidance of death drive or a desire to enjoy the new experience of children etc?

Is there such an experience as true reward - maybe it is only really a degree of satisfaction of a drive. Why does something taste good - feel good. Can it continue to feel good, taste good forever? Why don't we just stay in the situation that is good forever? Lie in the sun because it is warm - either because one other drive takes over and needs satisfying, hunger, sleep etc. or pain sets in - sunburn, swollen stomach. In the case of a robot we need to have boredom set in and be the drive that overrides a pleasant situation.

I believe it impossible to have a sequence of outputs without intermediate feedback at each step in sequence. You can't learn to speak if you can't hear yourself speak - or receive the speech back via some sense or other. E.g. feel the vibrations.

Since doing is the sequential execution of previous attentions and outputs then to think about doing you need the attention operation next input or next output.

If we were to make any sequence that is recognized as having been experienced before cause a reflexive output we would never repeat previous sequences. The reflexive output should only be generated when boredom sets in or resulting from punishment.

It is not necessary to have an output for every input.

What is a clear, useful definition of Boredom? When your expectations are the same as your previous expectations and you are not doing something with a purpose.

Environment for adaptron, Location when moved in to gives off different letter. Adaptron can do a number of things such as move up, down, left, right. Maybe some walls like a maze so that when it moves in that direction it hurts itself. What is its goal, purpose in life? To explore the environment. (Yes, but what motivates it? 1st April, 1990) To learn all it can about the environment, what it can and can't do. To learn all possible effects of what it can do. To continually find differences - new things to see, to do. In a closed environment like this one it should eventually become bored. If one square or combination of situations produces pleasure then it must also become bored of this situation too. Next level of environmental complexity is one which changes. E.g. squares move to different locations - input changes when robot does nothing. This can happen when free agent is also roaming in environment and collision encounters can occur.

Undated Note

Model its environment so as to use the model instead of the environment. What's its final steady state? Maybe cyclic steady state - always changing or cycling.

3rd April 1993 - When your inputs are the same as your past experiences and your thoughts are the same as previous thoughts and you are not doing something with a purpose, at a level of concentration.

19th May 1986

When the expected occurs - is input - and you are not doing anything - > Bored - life is dull - reflexive output - in order to generate something new.

When the expected occurs - is input - and you are doing something but at lowest level of concentration. -> Idling along - no particular interest in what you are doing just better than not doing anything because something might happen - maybe this should only go for one output before it stops doing - It is very similar to automatic level of execution - good for repetitive types of actions. Need to keep on doing for it not to stop. It also stops whenever expected does not occur - something unusual occurs.

When the expected occurs - is input - and you are doing something intentionally and with concentration. -> This is good - confirms the course of action. If unexpected occurs - surprise - halt action - rethink - chance to learn new things.

Expected means next thing when doing not necessarily associated. What is associated if it is not next? Next implies associated in time - can we have associated not in time? Probably not, the association has to be formed in the time dimension.

Since avoidance never becomes extinct - when we are about to generate reflexive output - When thinking is included - the reflexive output starts as an idea of output which should then be modeled - judged against past outputs before proceeding.

21st May 1986

[Muscles receptors referred to in 14th June 1985 are handled by cerebellum in man!]

We either must have a next input, next output type of primitive thinking function or associated input and associated output must be possible from a cue which is an associated input itself. E.g. think of a series of pictures from your house to work, each picture is pure thought. They are not actually occurring as you think so it is all thoughts. One picture is the cue and it stimulates the next recalled picture in sequence of motion to work based on sequence in time. If these are associated ideas (internalized pictures) then the cue idea must match the remembered original picture and generate an associated (idea) picture that act as the next cue.

Or the first alternative which is that we have a primitive function called get next picture which uses the localized location in context from where the last picture was recalled. This means no input information for a cue into memory to generate a recall.

Currently I lean towards the context oriented idea with a location that is active and get next is a primitive operation (attention operation).

However I also like the idea that the thought of an input matches with the recorded real inputs because this is the initial basis from which thoughts model reality. A thought of input is matched against our experience of reality and the results - associated outputs and rewards etc. are used to determine chosen actions. Thus our experience (recorded) acts as testing ground against which to try out our ideas-thoughts and get back what experience tells us will happen.

I'm getting the impression that thinking can take place in parallel with and independently of the input/output sequences. Attention is always paid to thinking when it occurs and only to input/output when learning is involved. It is as though input/output can go on from the built up habit base without attention except for the instigation of the habit.

And associations can be made between Inputs, Outputs and Inputs across any number of thoughts.

19th Oct. 1986

How many inputs/outputs do you process before boredom sets in if you have no purpose?

How many inputs/outputs do you process before your purpose (goal) is not realized?

In using past experience as a model for current behaviour at each input/output point a decision must be made as to whether you are bored- need reflexive output, going to get punished-need to avoid the repeat of the experience or to carry on because you will be rewarded or a goal will be achieved or to start doing because a goal is worth reaching, maybe rewarded.

So possible events & causes

 1/ Bored          -  reflexive output

 2/ punishment   - stop executing

 3/ reward         - start executing, continue executing

With the choice between the "get next primitive" and "the thought matches recorded input" we also have a third choice - This is where the thought always matches where it originated from and the associated what-evers come from that location - rather like an active memory trail - However this third choice might not allow for different but important memory trails to be recalled - only "the thought matches recorded input" would allow this out of context recall.

I propose  Idea of Input match Stored Inputs

               Inputs            match Stored Inputs

     but      Inputs do not match Stored ideas of Input.

This means that an idea of input immediately matches the stored input that generated it and therefore an active memory trail can be established but also other memory trails can become active and the trail acts very similarly to the next concept desired.

22nd Nov. 1986

Inherent in execution is the next input - next output type of operation.

A little clarification on input, stored input and idea of input is necessary. Input is the original signal coming in on a sense. This is stored in memory and is the stored input. A stored input is recalled and is then an idea of input. This is stored and is the stored idea of input which when recalled continues to be an idea of input.

Are thought and ideas actually stored for recall purposes? Why save an idea - maybe thoughts and ideas are really a process and the process is repeated to reproduce (recall) an idea - thought.

Why save an idea? Input - output is saved / memorized so it can act as a model database of reality so one can learn and think upon - it is experience itself. But ideas don't form a database as such for modelling! What about concepts - are they stored? - Take the concept of "impossible" - do we store the word and only really understand impossible as a thought process result.

The brain seems to be able to answer the question "to obtain this input what output do I need to do?" - reverse (in time) association - to achieve this goal what output do I need? (In this current context perhaps!) How does the idea of the goal arise - it must be recalled and linked by a recall process, which means it needs a cue that is the current situation - input thus context sensitive.

Need to implement:

 - Idea of Input match Inputs in memory

 - Idea of Input match Ideas of Input

 - Inputs          match Inputs

 - Inputs do not match Ideas of Input

 - Not doing and expected occurs (i.e. is input) (matches Idea of Input (associated Input)) generate reflex output, on unexpected - listen - use unexpected as cue.

 - Doing and expected occurs and no concentration then stop doing (bored) on unexpected stop doing - use unexpected as cue.

 - Doing and expected occurs and concentration normal then carry on, on unexpected occurs same as before.

 - Doing and expected occurs and concentration high - goal seeking then carry on, on unexpected occurs same as above. Any difference between this and previous?

 - Above analysis says do away with Doing and no concentration.

 - Expected when doing - means input matches the one executed.