1994

10th May 1994

What about making concentration into a continuous scale which is notched up and down based on feelings? Then interrupts would be on a continuous scale based on the strength of association or feelings.

I should catalogue all the human feelings and divide them into ones that are associated with learning and thinking and match them to logical events that occur in the mind. Examples are the feeling of reaching a goal, of failure, fear, mystified, lost, sad, happy etc. Then there are moods that are the states of emotion, which last a period of time as a result of feelings that are instantaneous events. What about confidence? The knowing that you can or the knowing that you know.

31st May 1994

Reactions to feelings - some instinctive things and some learnt things.

Some typical internal feelings:

Feeling of knowing the answer but recall is not providing the associations you need / want. => feeling of have a match but the particular association desired (attended to) is not available.

Feeling of I don't know about it. => feeling of no match found.

Feeling of last time I did that I got punished / rewarded. => controls thinking / behaviour - instinctive reaction can be overcome by learnt behaviours.

Feelings control the flow of thinking + behaviour.

Could put these instinctive reactions in memory so they can be overwritten based on learnt behaviour rather than hard-wire the instinctive reactions and not allow any override, or divide reactions into two classes: those that must be in program and those that can be in memory, e.g. reaction to pain in hard codes, reaction to "have no match in memory" in memory.

Feeling of Boredom and the ability to detect it and instinct (currently hard coded) to do random output when recognized could be put in memory.

Recognition of boredom is not simple as situations become more complex it is harder to recognize. This should be based on simpler criteria and trigger the feeling.

There should also be a set of feelings associated with S.T.M. processes. Maybe STM is easier to design if there are such feelings involved.

11th June 1994

An input sequence that changes without us affecting it means we are in an environment that is dynamic. Only the input following our output is a result of our output, e.g. D in the following:

 In ->   A B C A B C   D

 Out->               X

A sequence of outputs could be performed if there is no change in the inputs between the outputs.

 In ->   A A  B B  B  B  C

 Out->       X        Y   Z   M

then when same situation occurs again

 In ->   A    B        C

 Out->     X    Y Z M

Output done without waiting for input because it assumes it is the same background task would be monitoring inputs and only if it changed would it interrupt output sequence with the fact. Do we see this in infants?

A sequence of outputs without any inputs (should) imply we have learnt that only the sequence produces a change in the input resulting. How do we distinguish between just M producing C where Y and Z are superfluous and where only YZM produces C and Y & Z are important?

 In  ->    B   B

 Out ->    M

should be followed by the "unexpected" feeling or feeling of not satisfactory because no change occurred as was expected based on experience - 2nd B didn't match C and the feeling is disappointment - no change - bored. Avoidance of boredom means pursuit of change - exploration, so feeling of bored is stored to negatively associate this sequence so it is not repeated,

If this were the algorithm then the sequence B   B would have been associated

                                                                               Y

with boredom feeling in the above pattern.

Concentration is like a state that is changed based on certain events. - Maybe there are other states (moods) which get changed based on events (feelings) that occur and certain actions are possible / not possible when in certain states. State of wanting (goal seeking) for example.  Draw a state transition diagram!

Without the reward / punish coming from external inputs the reward/punish equivalents must be feelings of boredom(bad) and change(good) and these feelings must determine do / no do concentration level. So record the feelings of boredom and change.

In a dynamic environment boredom must also set in when repetition occurs.

We need a smile / cry output state indicator so the external world can react upon it. This is just like a mother who detects her child's mood by the way they act (cry, smile etc.) and then she reacts to this. On a robot maybe a primitive equivalent is a red (unhappy) and green (happy) light. Both lights off would be a possible state.

Reduce simulation to input / output / feelings - no expectations or ideas of things, only 2 levels of concentration - doing and not doing. Associations don't attract attention - they are just acted upon. Recall based on match of sequence of events.

1st July 1994

Some definitions of terms to get my mind around them:

Attitude: is a state of feeling (a type of mood?) associated with (relative to) a particular subject or situation. It's the feeling component of your opinion about something.

Goal: an objective or aim which is the idea of a state of the external world and / or the internal mental world that is desirable. When the state is achieved and recognized (matches the idea) then the goal has been reached and a good feeling results (accomplishment).

Plan: The sequence of activities (responses/outputs) and events (stimuli/inputs) that you intend to perform and detect in order to reach a goal.

Belief: Ones understanding of the real world which is captured / represented in the mental model one has. Ones expectations of what will happen as a result of a certain output.

Words used interchangeably:

     Activity=output=response

     Happening=input=observation=stimulus

4th Aug 1994

After each recall there are a number of associations predicted. Events happening (event) which matches any one of these associations reinforces the sequence - continues at same concentration level. Therefore only stop concentrating if no match to any one of recalled associations (or should that be the next experience to be executed - sequential?). The advantages to this approach are that it allows us to match more than one possible association and it follows the same rules as associations. This is a clarification of the idea from 15th March 93.

Sp = Punishment Stimulus - pain, Sg = Reward Stimulus - good.

This scenario is based on Stimulus/response/feeling associations (non-thinking - no ideas).

  --- no---> --- no -->                         only associate one stimulus ahead, what to

S--ok-->S  S  S--ok-->R                expect. only associate one response ahead, what I did in the past situation.

  -- no -------> --- no ->

S--ok-->S--ok-->R--ok-->S  S   only associate one stimulus ahead of response

                                                       This is the effect of the response.

  --no-->

S1 S2   R   Sg  Fg S3 Feeling associated across one stimulus

           --ok-->                     This is the feeling effect of the response when done in this situation.

   --no-->

S S    R   S S         does not make sense.

The reinforced cause and effect sequence is what happens the first time stimulus S1 S2 occurs. When S1 S2 occurs again after S1 a person or a dog have an expectation of what is coming. The dog salivates after the bell=S1. It expects the sequence. This expectation is a thought - Stimulus idea, rather than a stimulus, response or feeling. In a non-thinking S/R/F simulation (just learning) this would not happen and only S2 would initiate the response.

If S3 is not associated with Sg of Fg then we can not use S3 as a form of symbolic reward/punishment. Is S3 the goal - should it be input before Sg or is Sg the goal? If Sg is the goal then only one input brings pleasure and only one input Sb is to be avoided. Are Sg & Sb the goals or is S3 the goal, or is the feeling the goal?

5th Aug 1994

From "Processing, Doing, Concentration" - 17th May 1986 - Thinking is the mental execution of a sequence of experiences - running the model - mentally and while this happens paying attention to the associations that are generated. So we don't need next input/output as a type of association - it is the process of thinking about something that happened and the ideas are associations spun off of this process (see 26th March 1990 Reminding-Recall)

When we are executing a sequence with the expectation of a reward or a goal state in mind when we achieve the goal state and we recognize it (it matches with past experience that we are executing) then we have the feeling of success and that match of good feelings (success) is a feeling we experience/record.

Boredom is a mood (state) that is achieved after a number of feelings of "the same" have been felt.

If we have a Stimulus/Response/Feeling simulation with no ideas stored the memory trace of S-R experiences is the model of the environment and we follow it religiously avoiding pain, obtaining reward when it is one stimulus away - never having expectations more than one stimulus away.

In our S/R/F simulation the recalled S, R or F must be used but not recorded. It's not the recalled S's or R's that would be used - it’s the recalled F's that would be used to start or not-start execution. Notes from 14th Sept 1993 pg4 indicate just the feelings associated with the food stimulus is necessary to change the behaviour of S/R/F simulation.

The association / recall that is to be used is:

If Stimulus input to memory then for each match with experience recall/associate only one of S or an R or an F whichever comes 1st.

If the recall/association is an R then look for any associated F for it based on the same rule below for recall/association from a Response input to memory.

If Response input to memory then for each match with experience recall/associate an S, an F or an S then an F. No R is associated and if found stops the association. See 4th Aug 1994 - "Learning" for more info.

So Feelings are associated over one Stimulus or one Response. Second S or R stops the associations. The first associated response is not recalled for a match if we already have a first associated stimulus.

From - 20 - before 13th May 1970 - Humans as babies, very early on, learn to control their eyes to obtain a change in their stimulus (visual inputs) so as not to get bored. Thus they learn the cause, effect relationship of visual stimulus and eye motion responses. They explore their visual environment. After getting bored with the non-changing environment they then get pleasure from changes in their visual environment not caused by them or caused by them other than by moving their eyes.

10th Nov 1994

We need to have a saturation of reward process such that continuous reward for a repetitive action saturates and no longer provides the reward. Only after some time (# of events?) does the reward feel pleasurable again.

See Punishment / Reward of this date about the pleasurable stimulus not generating pleasurable feelings after continuous occurrence. Some sort of decay curve involved such that the stimulus again starts to produce good feeling after a time.

Need termination of feeling generated by stimulus after it has occurred repetitively without intervening stimuli.

11th Nov 1994

From Integrating Reaction and Planning in a Heterogeneous Asynchronous Architecture for Mobile Robot Navigation, pp 70 - 74, by Erann Gat, SIGART Bulletin, Vol. 2, No. 4, Aug. 1991, there is the idea that an activity is initiated by an output and stopped by another output but continues in effect between the two outputs. This means that only changes in the activity are controlled by the brain. Sounds similar to the input idea where input is only recorded when a change takes place!

The similarity and/or combinability of execution's next and association's next is also raised in comment of this date in Learning section.

Should there be any recall process after a response. This would imply asking the question “what did I see?” after I did something. In an "S/R/F only" simulation maybe there is no recall of associated stimulus, it is more likely that you would continue doing and if the same stimulus occurs you carry on else if it’s different you stop doing. The expectation of what stimulus occurs next is in the execution process not the recall process. Execution and recall are similar in this way in they both deal with what is next. Are they combinable or should they remain distinct?

Maybe responses are recalled through memory before they are output to find any associated stimuli and feelings. The behaviour is then modified at this point not after the cue stimulus. This would be done rather than recalling all the associations of the cue stimulus.

S1 S2 R1 SG FG S4   -->   S1  S2  R1i FGi  R1  SG  FG  S4

or

S1 S2 R1 SB FB S4   -->   S1  S2  R1i FBi  S4

SG is a stimulus that is good,

SB is a stimulus that is bad,

FG is a good feeling

FB is a bad feeling

R1i  is the idea of the response # 1

FGi  is the idea of the feeling good

20th Nov 1994

I tried the simulation from SIMULATION - IDEAS dated 5th Aug 1994. The activity is very similar to the behaviour of an ant. There are no forward associations or expectations of entering into a situation which may lead to punishment or reward until you are experiencing the last input that precedes the good/bad event. It does not do a very good Pavlov's dog simulation. Its intelligence level is more reflexive / instinctive. Model has been named INSTINCT.SAV.