5.0 Analysis - The Adaptron Model

5.0 Analysis - The Adaptron Model

Adaptron consists of an information processing theory of learning and thinking. This is a model, a conceptual understanding of the components and processes involved in learning and thinking. The model results from analyzing the subjects documented in Chapter 3.0 - Facts and Observations . Since it is a theory it includes parts that are not completely supported by the evidence. It is certainly not correct in all particulars. However, a concise, consistent, clear and complete theory of operation is necessary if Adaptron is to be built to meet the general goal of an intelligent agent that learns and thinks. The Adaptron model as presented in this chapter is only one of many possible models of learning and thinking based on the observations and facts.

This analysis draws upon the psychological results in Chapter 3.0 plus work that has been done in the fields of Artificial Intelligence, Cognitive Science, Robotics and Artificial Life. These fields have proven to be a rich source of concepts for this analysis. However this analysis goes further than many theories in that it introduces some new ideas and combines others that have not previously been expressed.

The Adaptron Model is structured similarly to Chapter 3.0. The subjects are analyzed in order from the more tangible and factually supported to the less tangible, observation or speculation supported. It is a description or model of the problem that the Adaptron Design must be built to solve. If Adaptron is to be a correctly functioning intelligent agent then its architecture and design must address this model. The model is a description of a subset of the observations from Chapter 3.0. It is constrained to a subset of the observations because it is not meant to simulate animal behaviour, just those aspects necessary for learning and thinking. Also Adaptron need not have the same goals and motivations as animals because it does not need to survive as we do. We are the ones who will provide Adaptron with its goals.

The requirements may appear to be too sanitized. However this has been done on purpose to identify those aspects that lend themselves more readily to design and implementation in a computer program or because they have already been satisfactorily simulated by other research.

5.1 The Environment

For purposes of understanding learning and thinking we must use a deterministic environment because it operates according to some laws of cause and effect. Given some cause, described in enough detail and a given context, you can discover the effect and have a high degree of confidence that the same result will occur when the cause is repeated in the same context. It is only with these hidden but predictable cause and effect laws of the environment that an intelligent agent can learn and operate with any degree of success.

Similarly an agent needs a relatively safe environment to give it the time to learn and think. Whether the environment is discrete or continuous, fully or partially observable, dynamic or static and/or multidimensional is not as important as whether it is deterministic and safe.

An environment with a single dimension is all that is necessary to demonstrate learning and thinking. A linear environment of this nature would only allow for a single event to occur at one time and thus there is no need for Adaptron to be able to handle two or more asynchronous but simultaneous events from the environment. This has implications on the nature of habits as performed by Adaptron. It does not need to perform any actions in parallel and thus habit performance can be done by the mind.

5.2 Stimuli and the Senses

Stimuli were described as all the information being detected by the senses and received by the brain.

At the most primitive level of description a simple stimulus consists of three pieces of information,

the sense from which it originated,
the feature and
the value.

Features are also called properties, attributes, aspects or characteristics. The value is the final measure of the stimulus. A simple stimulus can not be decomposed into parts. Simplified examples of stimuli in terms of originating sense, feature and their value might be:

sight, colour - blue,
hearing, volume - 6 decibels and
touch, texture - rough.

A simple stimulus is first described as originating from a particular sense and then as containing a certain feature detectable by the sense organ. The sense and feature without the value for a stimulus provide the dimension of the stimulus, i.e. the dimension in the environment from which it comes. Two simple stimuli may come from the same dimension but differ in their values or they may differ on both dimension and value. The senses involved in identifying a dimension include both external and internal senses.

Objects and events in the environment are usually recognized by detecting a combination of many simple stimuli that come from the object or event. A particular apple image might have the values of red for the colour, apple contours for the shape and 10 centimetres for the size. A different image of an apple shares most of the same dimensions and some or all of the same values because it is the same type of object. However a different image of an apple must have at least one value different from the first else from the agent’s perspective it is the first apple. The resulting combination of simple stimuli is referred to as a complex stimulus.

The complex stimulus of the apple image is composed of simple stimuli all of which originate from the sense of sight. Other complex stimuli may be comprised of simple stimuli from different senses. For example the sight of a door closing in combination with the noise form a complex stimulus of the event. In general any simultaneously occurring combination of simple stimuli from various senses and with various features is recognized as a complex stimulus.

Absolute stimuli are those actually measured by a sense and relationship stimuli are those formed out of the difference between two absolute stimuli. Size is a relationship stimulus as is duration, relative position (above, below) quantity (more, less) temp -hotter, brighter, colour greener, volume - louder, pitch, and any other stimulus dimension that can be perceived as being continuous and linear. Time - duration

Summary

Stimuli are information input to an animal's brain from its senses.
Simple stimuli consist of originating sense, feature and value.
Complex stimuli are combinations of simple stimuli.
The dimension of a stimulus is the originating sense and the feature.

5.3 Responses

Responses were described as the information sent to actuators to cause actions in the environment.

At the most primitive level of description a simple response consists of two pieces of information, the actuator, and the value. Examples of simple responses in terms of muscles and their values might be:

right eye right muscle, contract and
right biceps, relax.

A simple response is first described as belonging to a particular actuator and then what operation to perform, either contract or relax for muscles.

Animal actions however are usually comprised of several simple responses. These are combined at the same time to obtain the result, such as in closing one's hand. The resulting combination of simple responses is referred to as a complex response or a response pattern.

Summary

Responses are the outputs from an animal's brain to its muscles.
Simple responses consist of actuator and value.
Complex responses are combinations of simultaneously issued simple responses.

5.4 Behaviour and Reflexes

An agents behaviour is made up of many stimulus-response sequences. Each sequence consists of:

one or more initial / triggering stimuli,
zero or one response and
(zero or more resulting feedback stimuli. )

It is possible to have many initial stimuli with no associated response when no behaviour is exhibited. But in each behaviour sequence that has a response, the response is associated with at least one triggering stimulus which is contained within the initial stimuli.

Behaviour diagrams are used in this theory to illustrate how the processes of learning and thinking work. These diagrams use a symbolic notation to represent the stimuli and responses as stored in the memory of the agent. Each stimulus is an S and a lower case letter that makes it unique. Each Response is an R and a lower case letter that makes it unique. For example,

Sa, Ra [plus worded description as necessary]

means that when the stimulus a is recognized the response produced is a. Just because a is used in both stimuli and response does not make them the same, one is a stimulus and the other is a response. This particular example may represent the head turning reflex of a baby. Sa is the stimulus "Left Cheek detects pressure" and Ra is the complex response that uses neck muscles to turn the head to the left. If required the worded description will give an example of the behaviour sequence.

The , symbol represents the flow of time and that the agent has a memory of the stimuli and responses in this order. This is also the order in which the reflex gets performed when the reflex is triggered. [describe performing a reflex stored in memory]

The pupil size controlling reflex is made up of two such behaviour sequences. One reduces the pupil size when the light is too high and the other increases the pupil size when the light level is too low. They would be represented by:

Sb, Rc (bright light then contract iris)

Sd, Rr (dim light then relax iris)

[missing any explicit goal - built in to Sb and Sd]

A behaviour sequence can begin with one or more initial stimuli. This is shown by a sequence of stimuli in a specific order separated by commas before the response is achieved. For example:

Sa, Sb, Sc, Ra

If feedback stimuli occur after the behaviour sequence then these are diagrammed as:

Sa, Ra, Sb, Sc, Sd

where the ─> symbol means the environment has produced the three stimuli after the response Ra. Whether Sb, Sc or Sd is an effect resulting from the cause Ra is up to the agent to determine.

5.5 Recognition

Development of pattern recognition software is an advanced science with many decades of research supporting it. Techniques and rules for image and speech recognition are well known. Good computer algorithms exist for classifying things according to the multidimensional space represented by the many possible features. Neural net software has also proven to be a powerful tool in performing pattern recognition. Recognition is a mature science.

From a diagramming perspective recognition means that a high level stimulus Sa may be the result of recognizing a complex stimulus composed of three stimuli Sb Sc and Sd which repetitively reoccur co-incidentally or in this sequence but are not shown on the diagram. The recognition process has labelled the high level stimulus as Sa. Sb, Sc and Sd are the features of Sa and would be used by any pattern recognition software as the different dimensions of Sa. Since the recognition process is always active and has parallel access to all the stimuli in a given behaviour diagram, any stimulus received can be found in the diagram if it is familiar. A matching stimulus does not have to be on the left end of a sequence to be recognized.

(Sa = Sb,Sc,Sd)?

5.6 Habits

Earlier I defined a habit as a group of one or more behaviour sequences that has been learnt. Here I wish to broaden its definition to: A habit is anything that has been learnt and can be repeated subconsciously. This means that the recognition of a familiar stimulus is a habit.

Two kinds of habits are Stimulus Habits (S-habits) and Response Habits (R-habits). S-habits are those that only involve the recognition of stimuli. R-habits involve one or more actions / responses. S-habits, once learnt, are performed totally subconsciously. However R-habits once learnt, are always triggered consciously and then performed subconsciously.

The behaviour diagram for a one-shot habit that has been learnt for scratching an itch might look like:

Si ═> Rm -> St ═> Rs -> Sf ═> Rs -> Sf ═> Rs -> Sf ═> Rr

where Si is the initial itch,

Rm is the habit performed to move the index finger to the spot which is itching,
St is the touch of the finger on the right spot,
Rs is the scratch motion,
Sf is the feeling of the scratching and
Rr is the removal of the finger from the spot.

This habit is one which scratches the spot three times.

A continuous habit behaviour diagram will contain a stimulus and response pair at its end which is also at the beginning thus describing a loop for repetition of the habit. An example for clapping is:

Ro -> Sw => Rc -> Sc,Sn => Ro -> Sw

where Ro is the response pattern to hold up and separate the hands
Sw is the feel of the hands wide apart
Rc is the clapping of the hands together
Sc is the feel of the hands colliding
Sn is the clapping noise

On a complicated habit behaviour diagram a group of one or more sequences may be as complicated as:

Sa ═> Ra ─┬─> Sb,Sc,Sd ═> Rb ─┬─> Se
├─> Sf ═> Rc ──────┘
└─> Sg ═> Rd ─> Sa ═> Ra
Sh ═> Re

This diagram shows how two unconnected sequences belong to the same group, how the environment might produce a variety of possible effects from one response and multiple feedback stimuli are needed before a response can occur.

5.7 Attention and Concentration

Attention is the single channel through which stimuli must pass to be recorded in memory. This means that the stimuli and response comprising all behaviour diagrams, except for reflex diagrams, are the result of attention. There is no indication in the diagrams as to whether the attention was attracted or controlled when it was being recorded, and there is no need for such information.

The stimulus information that passes through attention for recording in memory is the stimulus dimension (sense and feature(s)) and the value that is detected at the instant the change in attention occurs. For example, when you hear the chimes of a clock the stimulus is the tone and volume of the chime when the event first attracts your attention. When you purposefully change your attention by controlling it to focus on a particular feature then the value of that feature or stimulus is recorded. E.g. when you look at the colour feature of an apple its value which may be a particular shade of red becomes conscious. This technique of using a change, attracted or controlled, as the trigger for detecting a stimulus works well for the creation of discrete symbolic representations of things that exist in a continuous environment.

Attention is attracted by either an immediate change, an historic change or by a relevant stimulus. Immediate change detection can be performed quite simply by comparing one stimulus with the next using attention memory. However historic change detection requires comparing the flow of stimuli from all senses with those in long term memory. This is a continuous subconscious process that our mind performs so that familiar stimuli, which comprise the majority of stimuli, get ignored while the unfamiliar stimuli do not. This process of ignoring familiar stimuli is a habit.

From a behaviour diagram perspective historic change and its attraction of attention work as follows. A typical behaviour diagram contains many related stimuli that can occur in sequence such as:

Sa, Sb, Sc

This is a sequence of related stimuli appearing in a familiar order. If it is also familiar that these stimuli occur in different orders then each order has been experienced and must be known. In this case the behaviour diagram would contain:

Sa, Sb, Sc

Sa, Sc, Sb

Sb, Sc, Sa

Sc, Sa, Sb

Sc, Sb, Sa

One possible sequence Sb, Sa, Sc is missing because the agent has never experienced it. So when the habit of ignoring these familiar stimuli is being performed and Sb is sensed the expected next stimuli are Sc, Sa. If it is Sa, Sc instead then this is unfamiliar and the habit will attract your attention to this novel situation.

Relevance of a stimulus is also a criteria for the attraction of attention. But the relevance of a stimulus is determined by the association of a stimulus with a feeling. Feelings include those of good and bad which are caused by pleasant or unpleasant stimuli. But familiarity and novelty are also feelings, not necessarily as intense as good and bad feelings, but feelings nonetheless. This fact allows the statement of a more general theory about what attracts attention. Attention is only attracted by relevant stimuli. This is broad enough to include immediate and historic change of stimuli because they are relevant based on the feelings of familiarity and novelty.

Concentration is a two state property of attention; it is either on or off. When concentrating attention is being controlled and neither familiarity nor novelty determines which stimuli are attended to. However stimuli that are relevant because of their association with good or bad feelings can attract attention when concentrating. When concentration is off the attention can be attracted by novel stimuli and good or bad stimuli.

The recognition process that matches stimuli with memory to identify familiar or novel stimuli is a subconscious and continuous process. It is one of the fundamental components that make up Adaptron. Each stimulus must be recognized at the three memory levels of reflexes, attention and long term. If it matches a reflex the reflex is triggered. If it matches long term memory it is familiar and ignored. If it is does not match long term memory it attracts attention. What recognition does when a stimulus matches attention memory depends on the level of concentration. If concentration is on then novel stimuli can not interrupt it.

5.8 Conscious Action

Habits are initiated by a conscious command response within our mind that identifies the habit to be performed. When a habit is begun the initial position of the muscles and limbs could be anywhere. Since responses are instructions to start or stop an action they describe changes that are to take place rather than absolute results and can be given to a muscle in any position. Given a starting position and a change to be applied you have a process which is the opposite of differentiation. The mathematical term is integration. Given the description of a change and a starting position you can determine what the actual motion and positions will be. Continuous sensory feedback can then be used to dynamically vary the responses and detect completion.

Another way of thinking of the performance of subconscious habits and conscious actions to initiate them is to view them as two parallel processes. The conscious process is fed stimuli by the attention and is involved in learning and thinking while the performance of habits is a parallel subconscious process. The term 'mind' is appropriate for our conscious processor and I will use 'sub-mind' for the parallel subconscious processor that performs habits. I like to think of the cerebellum as the part of the brain that provides the sub-mind, a second 'cortex' that is just responsible for R-habit performance while the cortex and other parts of the brain are responsible for the attention and conscious actions. However this analogy is not necessarily accurate, it just helps to emphasize the parallel processing nature of performing habits. In fact subconscious habits include reflexes and these are performed in the brain stem and lower parts of the nervous system and not just the cerebellum. So if humans have a sub-mind it is distributed to many levels of the nervous system. The mind and sub-mind work together smoothly with the mind delegating habits to be performed to the sub-mind and the sub-mind attracting the mind's attention when habits are completed or fail.

The sub-mind is actually a multi-tasking processor because it can be given commands to begin habits while it is performing other habits provided the different commands do not conflict in the use of muscles or senses.

xxxxxx performing habits - detail xxxxxxxx

Since the performance of a sequence of responses requires the continuous monitoring of sensory feedback for its successful performance and completion the memory representation for a habit is like any other behaviour sequence. It consists of a chain of responses and expected stimuli. The sub-mind must interpret this chain as a series of instructions that it must follow to reproduce the habit. As it comes across each response it either sends it to the appropriate actuator or recognizes it as a sub-habit to be performed. As it comes across each expected stimulus the sub-mind monitors the same sense and features (dimension) to obtain the feedback stimulus. Monitoring a sense is equivalent to what is done when you control your attention but it is being done by the sub-mind. Since the sub-mind is operating sub-consciously none of the responses it issues or feedback stimuli it detects are conscious or memorized.

As the sub-mind monitors feedback stimuli it compares them with expected stimuli. It continues if they match. If they do not match the sub-mind stops and returns the mismatched feedback stimulus to our attention for conscious interpretation. After each matching feedback stimulus the sub-mind recalls the next response. It performs it if there is a next response and stops if there is none. When it stops the last matching feedback stimulus is returned to our attention for conscious action if there is any.

(And since we have a hierarchy of habits based on lower level habits or responses any response in a response sequence may be the command to perform another habit.)

At any time a habit's performance can be interrupted if another command from your mind to perform a different habit uses the same actuators and/or requires the same senses being used in the first habit. Our sub-mind must work in this fashion to allow us to make conscious improvements to a habit as it is being performed. Unlike the computer keyboard and disk drive example when the second habit completes our sub-minds are unable to automatically resume the first habit from its point of interruption. A conscious decision must take place for this to happen.

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Summary

Conscious actions are the internal responses that begin or stop a habit.
Responses describe a change to be performed, not the final result.
Performing a habit uses feedback stimuli to monitor success.
Performing a habit involves attending to stimulus dimensions and repeating responses.
Habits finish because they succeed, fail or are stopped.
When a habit finishes the feedback stimulus may attract attention.
The mind is the processor that performs conscious actions.
The sub-mind is the processor that performs habits and monitors feedback stimuli.
The sub-mind can perform many non-conflicting habits in parallel.

5.9 Memory and the Mind

The types of information that must be stored for this to happen include:

stimuli (dimensions and values),
the order in which stimuli occur,
responses (actuator and value), and
the order in which command responses were issued.

REPEATED here from chap. 3.0

Because reflexes, habits and other behaviour require different types of processing there are appear to be three kinds of functionally distinct memories.

- Reflex memory - the memory of reflexive behaviour sequences for purposes of recognition at the subconscious level to trigger and perform reflexes.

- Short term memory - the memory of recent stimuli from the senses so that the attention can be attracted to any significant change that occurs.

- Long term memory - the memory of non-reflexive behaviour sequences. This memory is used for performance of habits by the sub-mind, recognition of historic change and relevance by the attention, and recall of the sequence of attended to stimuli and issued command responses. This memory keeps a diary of what you experienced and did.

END Repeat

Responses are not only stored, they are found and recalled for the purpose of reproducing previous actions. So response memorization involves three stages, the last two of which are subconscious.

The first stage is the storage of the command response that initiates a habit. When we issue a conscious command it must be stored as part of our experience so that given a similar situation we can repeat it, provided of course that it produces desirable results each time it is repeated.

The second stage is to search the memory and find the most recently stored command response and associated habit behaviour sequence. When a conscious action issues a command it must be matched with the associated behaviour sequence which comprises the habit so it can be performed by the sub-mind.

The third stage of memory for responses is more like the video recorder playing its image on a screen. It is the ability to retrieve the recorded behaviour sequence so the sub-mind can perform the habit.

REPEATED here from Chap 3.

At the beginning of this section we listed a number of types of information that must be represented in some form of memory and we identified some of these memories based on a functional categorization. Memories also have different retention properties depending on their purpose.

Reflex memory provides the ability to recognize stimuli and perform reflexive actions and must be built-in at birth and exist for our lifetime. These are resident in the spinal cord, and brain stem. They are innate and never replaced because you would not want to stop being able to recognize the heat of a flame or to forget how to blink.

At the level of your attention there must be a memory for the detection of immediate changes. This memory must be recording stimuli from all senses simultaneously but with a very limited time span. It must hold the stimuli for as long as necessary to detect a change from one stimulus to the next which is intense enough to override the level of concentration. This memory detects changes in the stream of stimuli before they become conscious.

In long term memory there is a record of the response and stimulus sequences for habits. Their lifetime lasts until they are replaced by more recently learnt ways of performing the habits. Replacement is analogous to forgetting but in this case it is re-learning that causes the forgetting. The use of memory in this way has been called the memory of skills by psychologists and is analogous to production memory in production systems as described in artificial intelligence.

At the conscious level the long term memory lasts a lifetime for most facts and events but unfortunately forgets old information, information of lesser importance or information that is less frequently used. At the subconscious level the long term memory is being continuously used. It is being used by the attention in two ways; for recognizing historic changes and for determining relevance, because both can attract attention.

END repeat

For recognizing historic changes the first stage of memorization, which is recognizing a stimulus, is used by our attention. In order to attract attention to your newly painted front door long term memory must be accessed. And then after you become accustomed to the new door colour which gets stored in long term memory your attention is no longer attracted to it because it is familiar. Thus all sensory stimuli must be continuously and sub-consciously compared with long term memories.

For recognizing a relevant stimulus the continuous and sub-conscious use of long term memory is also required. A stimulus is relevant if it is directly or indirectly associated with a feeling in long term memory.

Summary

Memory is required for reflexes, recognition and habit performance.
Memory maintains a record of behaviour sequences.
Three functional memories are reflex, attention and long term memory.
Three stages of stimulus memorization are: storage, recognition and recall.
Three stages or response memorization are: storage, finding and recall.
Memories differ in information retention time and functionality.
Long term memory is used for recognition and attraction of attention.

5.10 Goals and Motivations

A man made agent should not need goals that are as extensive as man's. It will not be based on biological components so should not require food, water or elimination of wastes. However it will need some form of energy and maintenance.

As a general rule the goals of a man made intelligent agent should not contain any goals that supersede those of man else we threaten our own survival. But the goals should be such that it assists us achieve our goals. Thus an agents goals should be a subset of mans. It should be primarily to learn and think. It should not have any long term survival related goals such as reproduction. However it should be interested in protecting its body from damage that would stop it from sensing or responding to the external environment.

Acceptable goals for a man-made agent are:

Interest in new stimuli
a need for energy replenishment
a need for maintenance
protection from damage
desire to do as man requests

5.11 Pain and Pleasure

In order for the agent to protect itself from damage some senses such as those of touch could produce pain. A pain level of a stimulus is pre-programmed to trigger specific avoidance reflexes.

5.12 Feelings and Emotions

In the analysis of emotions and feelings I am covering uncharted territory. The approach is based on the fact that there is information in feelings that is used in the thinking and learning processes. The objective is to identify which feelings are caused by which events and how the feeling information is used or interpreted in the performance of learning and thinking.

Emotional feelings are difficult to understand. Three reasons for this difficulty are:

their cause is not always obvious,
we usually react immediately as a result of them because they directly influence our behaviour and we don't stop to experience or think about them like we might when listening to a tune and
when we experience them there is no external sense organ involved in their detection as in sight and hearing, they just come into our minds.

Feelings are another kind of internal stimuli. They are stimuli because we detect them, we become conscious of them and we respond as a result of them. They attract our attention and we can remember them just like any other stimuli. They are certainly not something we produce like a response to cause an action in the environment. Feelings are a kind of stimuli that originate from an internal sense and are based on something happening to us.

One way of understanding this is to think of all the stimuli from all your senses, external and internal, independent of whether you are paying attention to them or not, as passing through an internal sense organ which generates appropriate feeling stimuli if it detects certain stimuli. Most of the source stimuli have no effect and produce no feelings but a few are immediately interpreted as pleasant or unpleasant because they are related to survival.

The most basic feelings produced are the feelings of good and bad. A pleasant stimulus will produce a good feeling and an unpleasant stimulus will produce a bad feeling. Because of the importance of these two feelings once they are produced they immediately attract attention no matter what our level of concentration. It is said that "You can never have enough of a good thing." However, you can because the pleasantness decays over time. Maybe the pleasantness is found in the change in feelings not the absolute value.

(Good and bad are interpreted as success or failure when associated with behaviour.) Our mind is also an internal source of feelings that it generates in the process of learning new behaviour. These feelings are those of success and failure. These feelings are the source of the emotions of pride and humility. These terms are used because from an information processing perspective they highlight the purpose of the feelings. Note that these terms were used when describing the performance of a habit. The sub-mind can detect a failure or a successful completion. Here the feelings are being produced by our mind when the successful or failed feedback stimulus becomes conscious. (are these really feelings or an interpretation of an event? maybe they are just reward and punishment i.e. good and bad in context of habit performance.) (Success means that the expected stimulus and actual stimulus are the same.)

Novelty is also the feeling that is experienced when a habit fails because the feedback stimulus is unexpected. The sub-mind detects the difference but the novelty feeling is generated by the mind. The returned feedback stimulus that was different from the expected one must be attended to, recorded and recognized. If it is a new stimulus then novelty is the feeling. (is recorded? and concentrated on). If however, the feedback stimulus is familiar but new in this situation then failure is the feeling that is recorded. Recognition may identify any alternative successful response that could be used in the given situation. Failure of a habit by the sub-mind is not a catastrophe. It is an opportunity to learn, to practice.

It was mentioned earlier that feelings are stimuli and they have many of the same properties of external and other internal stimuli. There is one other aspect of external stimuli that has a parallel with feelings. Stimuli are detected when a change occurs in the environment. Can feelings be said to occur when a change happens to ones emotional state? It is more likely the other way around. Your emotional state, your mood, is affected by your feelings. Your mood is caused by an accumulation of numerous feelings. For example, boredom is a mood that is achieved after some unknown number of feelings of familiarity have been felt. The following table aligns feelings with emotions, moods and some common natural responses.

Feeling Mood Emotion Possible Response

Source: The Senses

Good Craving Desire Pursue
Bad Disgusted Disgust Avoid
Reward Happy Proud
Punishment Unhappy Hurt
Uncontrollable Frustration/Angry Anger Rage

Source: The Mind

Success Happy Joy smile
Failure Unhappy Sadness cry
Novelty Attentive Surprise interest
Familiarity Boredom Acceptance

expected success Excited/Anxious Joy
expected failure Worried/Afraid Fear flee, avoid
expected novelty Interested Curiosity interest
expected familiarity Un-interested Acceptance

The following table summarizes the feelings, emotions and corresponding moods

Feeling Context Emotion Mood

Good after response joy, rewarded, proud happy, successful

after stimulus desire craving

Bad after response sad, punished, hurt unhappy

after stimulus disgust

Novelty surprise interest

Familiarity acceptance boredom

Feeling Mood Emotion Possible Response

Good stimulus Craving Desire Pursue
Bad stimulus Disgusted Disgust Avoid
Reward response Happy Proud
Punishment response Unhappy Hurt
Uncontrollable Frustration/Angry Anger Rage

Success habit Happy Joy smile
Failure habit Unhappy Sadness cry
Novelty Attentive Surprise interest
Familiarity Boredom Acceptance

expected good hopeful
expected bad anxious
expected reward
expected punishment fear
expected success Excited/Anxious Joy
expected failure Worried/Afraid Fear flee, avoid
expected novelty Interested Curiosity interest
expected familiarity Un-interested Acceptance

emotions

Anger - unexpected happens - not what wanted - unsuccessful when trying, extreme displeasure.
Fear - expecting bad thing to happen, expecting bodily harm, impending danger, something about to happen that is not wanted
Sadness - unhappy, compare what happened with expected - not wanted, sorrow, loss of good
Disgust -strong aversion, repulsed, repugnance, indignation, wanting to avoid
Joy -pleasure, gladness, satisfaction, success, causes delight
Acceptance -not wanted but will tolerate, the way it is, belief, approve
Surprise -unexpected happened, astonishment
Curiosity. -desire to know, inquisitiveness, want to learn, strange, surprising, odd, different from known
love, joy towards other person, desire of person

desire, want of object
happiness, joy
anticipation, expected good thing to happen
contentment, acceptance wit current state
satisfaction, joy of accomplishment, having something
pride, happy about oneself
humour, amused, good mood, laughing
enjoyment, obtaining pleasure from stimuli
reverence, think highly of other person, they have good self model
hope, wanting good thing to happen
awe, reverent fear or wonder, admire

impressed, affected or influenced deeply

determined resolved about the course of action, resolute, concentrating

belonging good feeling resulting from oneself being part of group

admiration pleased contemplation, wonder, respect, warm approval, regard with pleased surprise

mystified can’t come up with the associated idea – recall the smell of a telephone poll

emotions that are mostly unpleasant such as:

hate, loneliness, anxiety, nauseated, sorrow, indignant, remorse, pity, terror, bitter, embarrassment, shame, jealousy, frustration, contempt, mystified, lost, guilt, envy, scared and grief.

At the reflexive and sub-mind levels feelings do not exist. However the sub-mind may be a source of feelings. Decisions at the subconscious level are black and white. If the stimulus matches then the response is performed. But our mind is involved with achieving successful behaviour as measured against our goals and it is the feelings that provide our mind with a measure of how well it is doing.

(emphasize that feelings are another kind of stimulus, they are recognized, attract attention, stored and recalled. There are reflexive behaviour sequences triggered by feelings - e.g. fear)

Summary

Emotions consist of feelings, physiological responses and habits.
A feeling is the component of an emotion that is detected internally.
Feelings are stimuli.
Feelings are produced by an internal sense that interprets other stimuli.
Good and bad are two fundamental feelings.
In the context of a response, good and bad become reward and punishment.
Good and bad are interpreted as success or failure when performing habits.
Familiarity and novelty feelings are important in learning.
Moods result from an accumulation of feelings.
Memories with feelings are more likely to be recalled.
Feelings are used as feedback by our mind for learning and thinking.
Feelings determine our level of concentration.

5.13 Learning

5.13.1 Learning to Recognize Stimuli

Related stimuli

Learning to recognize related stimuli is the process of identifying things in the environment. One learns which stimuli are co-incidental with objects and which stimuli are the expected order of events. This form of learning relies heavily upon the feelings of novelty and familiarity that occur when a stimulus is memorized. As mentioned previously two stimuli that occur co-incidentally in a deterministic environment are very unlikely to occur again together unless they are related to the same object or event. Thus the three stage process that is described below by which we learn to recognize things works very effectively.

The first stage occurs when a simple stimulus is received that is novel.

The second stage occurs when a simple stimulus is received for the second time and is recognized as familiar.

The third stage occurs when the familiar stimulus is received the third time.

Features that take on values cannot be used for kind of identification. You can not say what kind of object it is that has a colour, a height, a motion, and a temperature. But stimuli with values can be used to categorize objects all of the same kind. E.g. the type of objects that have a blue colour, 2 inch height, stationary motion and. Need sub - object feature co-incidence without same values for purposes of categorization. What kind of object has a window, a door, a roof, a wall, and a chimney.

(working here as of 27 june 97 - 3 stages of memory switch so recognize, store and recall - novelty / familiarity in feelings worked on and is clear)

Need feature co-incidence without same values for purposes of categorization.

A common kind of change is caused by a movement in which one or more features change co-incidentally. We can recognize a known small segment that is reused in a longer segment as we are receiving it. Always interested in the novel parts of segment - they stand out in a long segment containing lots of known small segments. When listening (concentrating) on a piece of music we get pleasure (comfort) out of hearing the familiar - predictable / expected pieces. When listening to a new piece the novel parts (change in periodicity) give a pleasant surprise amongst the familiar parts. (use example of music as a sequence of stimuli which are either as familiar or novel)

Why detect familiar versus novel? - so we can not pay attention to familiar ones and get attracted to novel ones - habitualize the familiar ones - which are not relevant (not involved in or related to a S-R ) - no associated Good or bad feelings - We mark stimuli with familiar feeling and it never again is paid attention to and does not clutter up memory.(might stay in STM?) Mark it with novel the first time seen and it will go into LT memory. Only goes into STM if paid attention to it.

Need order recognition - expected in certain order versus co-incidence where always happen together recognition. (Recognition of the order of stimuli not important most of the time just as long as they are co-incidental. But order is some times important and then the recognition becomes more specialized.)

Change reflexively attracts attention but it may not be stored if its familiar. If it is novel then it is memorized and we pay attention to it.

Sub-mind also does sequence of attention controls that are familiar but the only reward from stimuli is novelty. Is sub-mind intentionally looking for novelty - is novelty the "reward" or is sub-mind in habit of doing recognition of familiar stimuli and consolidating them to come up with final highest level stimulus that will hopefully trigger a response.

Simple stimuli appear to pack into STM and we build a more complex 'image' in STM as more simple stimuli are entered and then we can be given a complex image idea and decompose it using STM of the attentional ability of our mind to observe details of a memorized image. Is sub-mind making this packing of the familiar simple stimuli unconscious and just returning the memory location of the final familiar item - stimulus. Yes! If the segment continues to be familiar (familiarity rating steady or increasing) as stimulus grows longer then it remains in sub-mind. As soon as familiarity rating decreases then interrupt consciousness with novelty. If familiarity steady or growing and response follows it then based on goal feelings may become conscious triggering stimulus. - Memories purpose is for recognition. STM's purpose is to recognize short term related patterns.

Triggering and feedback stimuli

We first learn to recognize things at a conscious level. It is at this level that we learn to discriminate between the many stimuli, their features and values. We consciously identify the one or several features and/or values of the stimuli that are critical for triggering successful behaviour. In psychology this is called discrimination learning. This learning relies heavily on co-incidence of stimuli. It is done when we are concentrating on a stimulus. As in learning new habits, feelings are critical in learning new stimuli. At a conscious level the feelings of novelty and familiarity are returned when a stimulus is detected.

It is a process of discriminating between some stimuli and deciding they are different while generalizing between other stimuli and deciding they are the same.

(We tend not to learn to discriminate between the many different kinds of snakes based on their markings unless we live in an environment where snakes are dangerous and thus we have a need to distinguish between them.)

(Squash and tennis habits tend to replace each other until enough practice is sufficient to increase discrimination of which triggering stimuli and which feedback stimuli are really the correct ones in each situation. Wrong ones cause us to be come conscious of them and we learn the correct categorization - learn to recognize the true triggering stimuli ).

(use an example of the head rotation of baby when finger on cheek to illustrate discrimination. i.e. touched with glove, ice cube, warmth. baby should learn to tell difference and not rotate for the ice cube).

(Unexpected stimuli are detected by the sub-mind execution comparing actual with expected feedback stimuli! When the attention is paid to it and compared to memory it may be truly novel or it may be recognized but it is in an unexpected context.)

concentration

level feeling meaning

none novel pay attention - response might be necessary - find out about world, relationship between objects / events - remember it.none familiar more of the same - bored, nothing to do, possibly try random act to see how environment affected - mark it with familiar feeling so attention is not attracted to it and we do not have to memorize it again.
high unexpected different from one being repeated - either novel or familiar
high novel lower concentration level - broaden pattern match - searching for possible response - interest is drive = avoid boredom
high familiar but not expected? not same as what was detected last time - if success then perform it or rewarded then repeat it and don't drop concentration level - weave it into this new habit being learnt - this caused a conscious interruption - else it just familiar but unexpected at this point - so its novel in this situation - have a next expected stimulus so idle.
high expected must be familiar. feeling of success - labels stimulus dimension as correct- confirms doing well - carry on

When concentrating we tend to flag those features that are common to the triggering stimulus and the past stimulus as familiar and those are the ones necessary to further trigger the behaviour. This then lays down a stimulus stream that identifies the situation that must be familiar for further repeat or performance of this behaviour. Familiarity consolidates during stimulus learning. Familiarity grows, is gradual as we learn new stimuli.

(The memory and its ability to learn how to recognize the triggering stimulus plays a significant part at the beginning of this scenario but this learning process is dealt with later in 'Learning New Stimuli'. The power of the mind comes from the ability of the memory to recognize the situation, the triggering stimulus and recall relevant past experiences from which the correct action can be chosen. Past feelings are used to make this choice because they provide the survival related feedback.)

5.13.2 Learning to Perform Habits

In the above examples the baby is beginning to learn about its environment and store the experiences in its memory. But if it is to use these experiences to change its behaviour it has to be able to interpret them and use them in making decisions about what to do the next time or in new situations. It needs to make choices about its actions. The process of using experiences and choosing actions implies there is some form of mental processor that is processing the experiential information. This mental processor is commonly called the mind. The mind is the seat of consciousness and is the processor that performs learning. At its front end it has the attention that it can control when it concentrates and at the back end it has the sub-mind to which it sends commands to begin responses. In simple terms, to perform learning of new habits the mind compares stimuli with experiences, repeats the rewarded experiences and avoids the punished ones.

To understand in more detail what is taking place in the learning of new behaviour let us use the scenario of a baby sucking on a finger for the first time. The following sequence of stimuli, responses and feelings make up the first experience passing through the baby's mind:

1. The triggering stimulus - the touch of the finger to its lips, gums and / or tongue,

The touch of the finger may have produced a combination of stimuli from lips, gums or tongue and have multiple features such as pressure, texture and heat but the baby's reflexive response may only be triggered by one of the features and by only a subset of the places touched, e.g. lips. The whole triggering stimulus (the combination of stimuli) provides the context for the situation in which one or more features are recognized and found to match the reflexive behaviour's triggering stimulus.

2. The reflexive response - sucking (the command issued to the sub-mind),

This is a command to begin sucking which is a looping sequence of response patterns involving the lips, tongue jaw and diaphragm. The sucking habit will repeat as long as its expected resulting stimulus continues. This is most likely the correct contact with the lips. The initial sucking command is conscious in that it is memorized but no conscious decision is made to start it. A reflex makes the decision and subconsciously originates the habit.

3. The feedback stimulus - the texture and / or taste of the finger.

While the sub-mind is performing the sucking the feedback stimulus will attract attention. The feedback stimulus will depend on the environment at the time that the reflexive response was performed. In other words the baby may taste honey, salt and/or a rough finger.

4. The resulting feeling - reward, punishment or novelty.

A good feeling of reward results from a sweet taste. This reward will not stop the sucking but may trigger an innate reflex to suck harder. Punishment results from the taste of salt. Either salt or the bad feeling of punishment will most likely trigger the reflex to stop sucking and eject the finger using the tongue. Novelty results from a novel texture that produces no feeling. Thus the sucking will continue. If the finger is so rough that it hurts then the resulting feeling is punishment because the resulting stimulus is pain. The pain will trigger the reflexive response to cry, which will interrupt the sucking because crying uses the diaphragm and mouth muscles. Neither familiarity, success nor failure are possible because it is the first time this experience has occurred. Although familiarity will set in after prolonged sucking resulting in a novel stimulus because the novelty will wear off.

Experience is made up of sequences of such stimuli, responses and feelings. These are the memory sequences representing our behaviour. Each sequence starts with a triggering stimulus, then a response issued as a command to the sub-mind. It finishes with the resulting stimulus experienced and resulting feeling.

Just like when the sub-mind performs a habit, when the mind repeats a previous response and stimulus sequence it interprets the experience as a series of instructions of what to do. When a previous successful response is repeated it is delegated to the sub-mind to perform. When a previous stimulus is repeated, the mind controls the attention to focus on the same sense and feature(s) as the previous stimulus. The concentration level for paying attention determines whether any other stimuli can interrupt the minds conscious repetition of the past experience.

Now let us consider what the experience will look like when the baby encounters the touch of a finger for the second time. This is assuming that the baby does not need to experience the first scenario a number of times before it progresses to this second level. This assumes the mind is a perfect learning processor and makes the most efficient choices based on experience. However this is obviously not true. We often have to experience the same thing several times before we learn it. This approach helps in learning cause and effect relationships which are not always deterministic, ones that have some degree of uncertainty to them.

The second experience begins when attention is attracted by the triggering stimulus. It consists of:

1. The triggering stimulus - the touch of the finger to its lips, gums and / or tongue,

Having attracted attention this stimulus is conscious and follows the other two stages of memorization; storage and recall. Recognition has already occurred sub-consciously in order that the attention is attracted. To be recognized in the first place, one, some or all of the features of this triggering stimulus must have been the same as in the first experience. However the recognized feature(s) does not have to be the one(s) necessary to trigger the innate reflexive response. After the stimulus is stored, recall is performed. The mind remembers the expected resulting stimulus and resulting feeling from the first experience. These are called the expected stimulus and expected feeling.

2. The expected stimulus and expected feeling - these are the expected sweet taste, unpleasant salty stimulus or novel texture.

At this point the mind has the choice of either starting to repeat or to not repeat the behaviour as recorded in the first experience. It bases this decision on the resulting feeling in experience one - the expected feeling. If the feeling was of reward then it repeats it at a high-level of concentration. If the feeling was of punishment then it stops in order to avoid receiving the unpleasant stimulus again. In order to stop the mind returns to a level of no concentration and waits for the next stimulus to attract its attention. If the feeling was of novelty then it repeats the past experience but at a lower level of concentration. Let us assuming the mind decides to repeat the past experience. 3rd and subsequent experiences - If the feeling was of failure it does not repeat its previous response, it returns to a mode of no concentration waiting for another stimulus to attract its attention. If the feeling was success then it delegates the rest of the experience to the sub-mind.

3. The response - sucking (the command repeated by the mind and issued to the sub-mind to perform),

Since the sub-mind has been given a command to begin a looping response it will never attract attention with a successful result. It will only attract attention if the sucking fails for some reason, e.g. the mouth is empty because the finger was taken away.

4. The resulting stimulus - the texture and / or taste of the finger.

While the sub-mind is performing the sucking the mind repeats the next step in the first experience which is to pay attention to the same sense and feature as previously. It will do this at the level of concentration established in step 2 above. The actual resulting stimulus will depend on the environment at the time that the response is performed. The baby may taste honey, salt and/or a rough finger. This may or may not match the resulting stimulus of experience one. (Another possibilities that may occur at this point is the failure of the sucking habit. This will attract attention away from the feature and sense on which it is concentrating.)

5. The resulting feeling - reward, punishment, success, novelty.

Again a good feeling of reward results from a sweet taste and it will not stop the sucking. Salt or a rough finger that hurts will result in punishment which will trigger the rejection or crying reflex respectively. The feeling of success results when the resulting stimulus produces no good or bad feeling but it is the same as the goal stimulus. The sucking will continue in this case. Novelty will occur when a new texture e.g. that of a glove, is detected. It will obviously be different from the goal stimulus and sucking will continue. (However, if the fingers texture as detected matches the resulting stimulus of the first experience then the resulting feeling is familiarity. It is only when this sequence is experienced another time that the familiar texture will be the goal stimulus and the result will be boredom from the feeling of familiarity. When that occurs this response sequence will not be repeated.)

If we repeat a response and the feedback stimulus does not match the expected stimulus it should be followed by the failure or novel feeling. The feeling of failure occurs because the feedback stimulus is familiar but not expected in this context. Three sources of feelings - Match/No-Match expected goal, feeling from actual resulting stimulus and Match/No-Match expected feeling.

The memory and its ability to learn how to recognize the triggering stimulus plays a significant part at the beginning of this scenario but this learning process is dealt with later in 'Learning New Stimuli'. The power of the mind comes from the ability of the memory to recognize the situation, the triggering stimulus and recall relevant past experiences from which the correct action can be chosen. Past feelings are used to make this choice because they provide the survival related feedback.

If we look at the feelings as originally experienced and then what actually happens we can see all the possible patterns of learning. The triggering stimulus is what identifies the habit.

In tabular form the following possibilities occur.

Expected Concentration Resulting

Feeling Level / Action Stimulus Feeling

Success Low or None /

Perform

The sub-mind becomes active performing the learnt habit that follows the triggering stimulus in memory. The mind is then free to lower its concentration and pay attention to any other recalls from memory or stimuli. The following is being done by the sub-mind.

Feedback Stimulus Action

matches expected continue

Not a reliable match alert mind (attract attention)

Familiar so re-evaluate resulting stimulus to see if it is part of a habit marked with success. If so start performing it. If no habit then lower concentration and broaden pattern match / recognition criteria.

Expected Concentration Resulting

Feeling Level / Action Stimulus Feeling

Reward High / repeat Matches expected Reward or Success?

(must be pleasant like first time)

No Match Pleasant - Reward

(different so decrease concentration?)

No Match Unpleasant - Punishment

(zero concentration)

No Match Novelty

(attention attracted - zero concentration)

No Match Familiarity

(decrease concentration - recognized)

Novelty Low / repeat Matches expected Success or Familiarity?

(predictable)

No Match Pleasant - Reward

(do it again next time)

No Match Unpleasant - Punishment

No Match Novelty ( random? )

No Match Familiarity

Failure Low / no action what process created this feeling?

Punishment Low / no action need to figure this one out

Familiarity Low / no action looking for more interesting things

repeated from section 3.13 xxxxxx

A successful experience if not rewarded continuously can become a boring one after any novelty has worn off. Also a novel experience will continue to be repeated provided new resulting stimuli are detected.

Most of the time the feelings of novelty and familiarity are the ones that cause us to repeat past actions. This is usually done at a low concentration level such that other novel stimuli can attract our attention. It is only occasionally that a pleasant or unpleasant stimulus occurs to give us the feeling of success or failure. Success feelings produce higher levels of concentration when the experiences are repeated.

xxxxxx

The sub-mind uses these chains of experience as long as it can find another practiced one. If we detect a familiar stimuli we either do the behaviour sequence as habit because its goal feeling was success, or we repeat the sequence at a concentration level because the goal feeling was reward, or we don't do them because their goal feeling was familiar or punishment. It is most likely that the associated idea of a successful stimulus that will trigger the behaviour and it will be relegated to habit performance. Thus this has higher priority than the familiarity feeling in choosing to proceed. We do this by aggregating commands as though they were responses. For example you can learn to pat your head with one hand and rub your tummy with the other hand. You build this from the two habits you already know.

The highest levels of commands are the ones our minds issue. We are conscious of these commands. The sub-commands and responses used in performing habits are conscious during their learning and practice but become subconscious thereafter. They once again become conscious when we need to relearn them such as when they are interrupted by a failure in a particular situation. When a failure occurs our mind becomes conscious (is interrupted by the sub-mind) the feedback stimulus which didn't match the expected stimulus. Our mind then has a context and stimulus that it can store in memory and recall any past experiences that will enable it to decide what to do e.g. a goal stimulus and feeling.

So learnt behaviour sequences are relegated to the subconscious after they are learnt and become conscious again only when and if they ever need to be relearned. All responses are always given to the sub-mind to perform. When does, what situation occurs such that the mind stops concentrating on the resulting stimulus and leaves this up to the sub-mind to match?

To combine two unrelated habits into a new habit requires the mind to concentrate on the recognition of the feedback stimulus that resulted from the successful performance of the first habit and use the feedback stimulus to recall another successful habit that it can delegate to the sub-mind without stopping to record the success feeling of the first. Take for example two chains that have proved successful in the past and are habits. The first is the movement of arms together so the hands bang into each other producing a clapping sound and the impact feeling from the hands. The second is the separation of the hands apart a certain distance so nothing is felt by them.

Both habits finish with a success feeling placed there when they were learnt.

After the first habit is triggered and performed by the sub-mind and reaches the end the mind is attracted to the feedback stimulus. It finds it to be a triggering stimulus for the second habit and it performs the second habit.

Learning about the relevance of a stimulus - read paragraph in attention/concentration.

Learning orienting responses as good example where no reward involved!

What about looping?

Unconscious activities

Since it is impossible to learn a sequence of responses without intermediate feedback stimuli at each step in the sequence a child can not learn to speak if it can not hear itself or receive the speech back via some other sense, e.g. feel the vibrations. Helen Keller? During learning the feedback stimuli are conscious and concentrated on by the attention. The feedback stimuli become subconscious after the learning is complete. The feeling of success created by the mind when the feedback stimuli matched the expected stimuli labels the response as a learnt one that the sub-mind can perform. If the feeling of failure is recorded then this experience chain will never be habitualized. It has been shown that when the feedback is taken away or distorted in some way the automatic process of speech becomes interrupted.

When sub-mind completes successfully it makes last feedback stimulus available to attention. This is the one that matched the last expected stimulus that was followed by the success feeling and the one it could not find in memory as a triggering stimulus for any other behaviour chain. This last feedback stimulus is less likely to attract attention than a feedback stimulus associated with a failed match. A looping habit is natural consequence of sub-mind searching for any follow-on successful behaviour chains.

still need answers to:

Maybe only can have a looping habit which is short enough to all fit in short term memory.

When conscious we stop repeating a behaviour when it becomes familiar - it becomes boring. The sub-mind however continues to perform a behaviour only when it is familiar. Somehow the idea of success keeps us doing behaviour sequences even though the recalled feeling is of familiarity!

Doing

19th May 1986 - When the expected occurs - is input - and you are not doing anything - > Bored - life is dull - reflexive output - in order to generate something new.

15th June 1985 - A sequence of stimuli and responses once learned does not become habitual if there is no associated reward or idea of reward each time it is done. So if there is a stimulus - response (unrewarded) sequence that is known it is not performed because it does not produce any new inputs i.e. curiosity is not satisfied (see below 16th Aug 1981). 16th Aug 1981 - We currently execute the past experiences even if it was not rewarded. This implies past actions taken were acceptable (not punished) but we are really not furthering our goal by doing this since we are not pursuing interesting things.

- When below concentration level of directed thought and action then any sequence that has been seen or done before causes lack of interest and a desire to change the world, which means reflexive output.

27th Aug 1976 - Possible reasons for stop doing: pain interrupts attention and sub-mind due to reflex. But pleasant stimulus may not interrupt either.

11th April 1993 - The phenomena of 8th June 1972 - 165 (this section) could be explained by the minds ability to directedly execute associated idea attentions but not directedly execute next frame attentions.

8th June 1972 - 165 - How is the following explained. I memorized a series of letters VNSBPX. After many minutes of doing other things I tried to think of what the series was. I could not remember the X yet when I was not trying to recall the X specifically but just rattled off the series I remembered the X easily. The rattling off is an automatic well learnt series of mental commands where as thinking was a process of producing the concept BP and trying to get the association X but I came up with S instead.

We also learn the orienting responses that increase our ability to detect the triggering stimuli. Then when we are subconsciously performing the behaviour as a habit the sub-mind's attention can focus on and recognize just those stimuli and features that are important for feedback purposes. We have learnt which features on which senses to which we must pay attention. For example, while driving, if it is sunny, I have learnt to pay attention to the shadows of the vehicles I am following if I can not see them because they are obscured by the vehicle directly in front of me. This provides advanced warning of vehicles slowing down. However, should the vehicle directly in front of me slow down my attention is attracted by its brake lights or because it becomes larger in my field of vision as it comes closer. At a young age we learn and habitualize the control of attention and the orienting responses required to enhance our stimulus input e.g. eye motions.

(We saw that attention was attracted to change because it provided for an opportunity to learn. Thus avoidance of boredom is the same as a desire to learn new and interesting things. Learn about how the environment operates, explore it for its hidden laws of operation. It is only natural that this is a goal of an intelligent animal because it is through better understanding of its environment that it can better satisfy its primary goals.)

(Two situations which are close enough? alike that the 1st triggers the second. Close enough is a learning of recognition - discrimination / generalization. In baby finger on lips 1st experience 6 features experienced 2 necessary to trigger innate reflex. Other 4 provide context. 2nd experience 3 features experienced - they match 3 of 4 context stimuli, none of which match the 2 necessary to trigger innate reflex but behaviour performed. Thus generalization. Specialization requires pruning of the valid stimuli - needs thinking about! )

5.13.3 Summary

Avoidance of boredom - pursuit of interest/change/differences. Does this mean that in the same situation as before does a robot experiment with new responses? Maybe it does if the old ones are not rewarded and not punished i.e. no feelings or idea of feelings obtained in the past situation. As long as it got an idea of reward then the sub-goal is being achieved in the past because the idea of reward feeling is obtained when the goal is achieved. For goal seeking to exist it needs at least one thing to have been rewarded. It needs this before it can come up with goals and sub-goals. Without this reward it is just trying to avoid boredom. Ideas of reward chain backward overtime through repetitive associations so that long chains of sub-goaled executions exist all focusing on at their end the rewarded state. S1 - R - S2(Rewarded idea) when recalled on input of S1 results in S0 - R - S1(rewarded idea chain sub-goal backwards one step.) Feeling of familiarity can only occur if you are concentrating on, paying attention to the stimulus because you do not get bored with the colour of your room unless you pay attention to it - it is familiar - you usually ignore it. Recognition of boredom is not simple as situations become more complex it is harder to recognize. This should be based on simpler criteria and trigger the feeling.

- When we are executing a sequence with the expectation of a reward or a goal state in mind when we achieve the goal state and we recognize it (it matches with past experience that we are executing) then we have the feeling of success and that match of good feelings (success) is a feeling we experience/record.

1st Jan 1973 - 178 - Pain as input does not have any built in association. Recall of input pain and recalled concept of pain have meaning. On recall of pain input the directions are not followed the thing is definitely not done. On the recall of concept of pain can carry on thinking and could actually do the thing the concept of pain does not stop one from doing a thing i.e. will over want. (Reason over will) (Mind over matter?).

The idea of pleasure i.e. recalled pleasure acts as a guide to thinking. Actual pleasure is what is wanted and nothing is done on actually receiving it. On recall of actual pleasure thing is done. This is a problem, order of recall = thing done, actual pleasure how do we get back to do thing done.

14th Sept 1993 good summary of learning - also learning to avoid. - A typical sequence of learning events resulting from rewarded behaviour propagation are:

S S R S F

bell food eat taste good

S= Stimulus, R=Response, F=Feeling, Fi=Feeling idea. Stimulus invokes the feeling, not the response.

S S Fi R S F

bell food good eat taste good

^ ^

execute halt

S Fi S R S F

bell good food eat taste good

^ ^

execute halt

Punished behaviour - propagation cancels learned behaviour:

S Fi S R S F

bell good food eat taste bad

^ ^

execute halt

S Fi S Fi S

bell good food bad ?

^ ^

execute halt

S S S

bell food ?

If there is punishment only:

S S R S F

bell food eat taste bad

S S Fi S

bell food bad ?

This scenario assumes agent in control of responses and stimulus is a result of the responses. There are some environments where you loose control - doing nothing results in punishment. How does this stimulate agent into doing something e.g. avoidance - Avoidance does not become extinct.

S S R S F R S F

bell food eat taste bad spit taste good

^ ^

controlled reflex

response reaction

or consider:

S S S F R S F

bell red-zone electricity bad leap green-zone good

S S Fi R S F

bell red-zone bad leap normal good

In these sequences we do not recall Si and then Fi. When do we produce Si recollections.

15th July 1992 - Some rules for learning compiled at AAAI conference and while listening to Case Based Learning by Kristian Hammond.

Don't make the SAME mistake twice. - make a different mistake and learn.
Try what has worked before in a similar situation.
The world is a safe, stable and calm place so you can reuse experiences over and over again.
If it worked in the past use it again, it will most likely work.
If it does not work - don't do it that way again and try something else.
Use the environment to test what works. Doing is more important than proving.
Combine small schemes to do hard things, learn to do easy things first in order to combine them for learning to do harder things.
Learning is acquiring the knowledge about what to do in given situations in order to achieve your goals and avoid problems.

I need to explain how we create sounds that are the same as the sounds we have heard in the past - The imitation learning process.

Summary

Learning is a process of improving behaviour based on experience.
Learning improves stimulus recognition and habit performance.
Both kinds of learning require memorization.
Improvement is measured through feelings.
Learning new habits involves practice done at a high level of concentration.
Learnt behaviour sequences with expected success can be done by sub-mind.

5.14 Ideas - Recall and Associations

5.15 Thinking

some of the items in Architecture belong in here!

10th Nov 1994 - We need to have a saturation of reward process such that continuous reward for a repetitive action saturates and no longer provides the reward. Only after some time (?, # of events) does the reward feel pleasurable again.

17th May 1986 - Is there such an experience as true reward - maybe it is only really a degree of satisfaction of a drive. Why does something taste good - feel good? Can it continue to feel good, taste good forever? Why don't we just stay in the situation that is good forever? Lie in the sun because it is warm - either because one other drive takes over and needs satisfying, hunger, sleep etc. or pain sets in - sunburn, swollen stomach. In the case of a robot we need to have boredom set in and be the drive that overrides a pleasant situation.

- Only in the Feel-Muscles class of input/output devices are their reward/punishment feedback available for a robot. Punishment may not be the best word to use because it implies outside influence by an active agent. Let us call it pain, A bump on the head from slipping on a shiny floor. Humans can get reward and pain from the mouth-taste class of devices where there is also the primitive hunger drive.