2002

17th Feb. 2002                     Inhibition

I have always thought the best thing to do when a negative response [stimulus] is received is to stop doing the habit or action being performed. This is the essence of inhibiting action (freeze or flight). The freeze is what essentially happens when one neuron inhibits another, it stops it from executing. And this happens at any level in the recognition action hierarchy.

I could play solitaire on my computer while talking on the phone. But playing solitaire requires some decision making so it can’t totally be relegated to habitual action sequences. Yet that is what it felt like. It felt as though I was doing even the thinking part of the game playing habitually. The movement of the mouse was definitely habitual when I played Sherlock though my conversation was interrupted by the need to think – make decisions.

16th June 2002         I started taking some notes on my computer

Notes and Ideas about thinking and learning

A binary neuron recognizes stimulus 1 followed by stimulus 2. Could call it a binon. More sophisticated binons could recognize stimulus 1 and 2 at the same time with no opportunity to notify (stimulate) two or more other binons thus no opportunity to have an associated response between the two stimulus occurring. This would correspond to the neurons in the eye which just learn to recognize – don’t learn to do.

Could also have a binon that normally recognizes 1 then 2 but is triggered by stimulus 2 and fires if stimulus 1 occurs simultaneously or very shortly thereafter. The thereafter interval to be set so short that performing any reaction between 2 and 1 is not physically possible.

Also could have a binon that recognizes an increase or decrease and it’s magnitude between two stimuli that form a continuous scale such as volume, pitch, colour?, angular displacement (horizontal, vertical) in the eye.

We learn to recognize and we learn to do.  Learning to recognize is the combination of SRS into one new S (a recognition habit) where the R can only be a response habit (automatic / subconscious) that adjusts our sensors to help obtain the second S.  Learning to do is the combination of RSR into one R (an action habit) where the S can be done / recognized totally automatically / subconsciously.

When we get a first S and consciously want to obtain the second S we do the intermediate R. To do that R we ask the subconscious / cerebellum to do it. [Or do we do it consciously / concentratedly / with practice first and if successful (achieves desired result – the second S)]  The subconscious starts doing the R and waits for the resulting S and if it gets it and there is another R it does that too. It does not check ahead for desired outcome. No, the subconscious is given the higher level R and it does the RSR it is composed of. It will only cause an interrupt of consciousness if the S does not occur.

The process of practice builds the RSR response / action habits and the SRS stimulus / recognition habits.

Recognition habits form provided we are conscious of the SRS sequence and the second S (goal) is accomplished / experienced. New S’s can act as a goal based on the idea of novelty. Given another S1 we will repeat R and S2 because S2 was successful last time. The S1RS2 is a recognition habit.

Now, how do Action habits get formed?  The R is the recognition habit could be a primitive R or an action habit. If it is an action habit then it used to be R1 S3 R2. What was the practicing / concentration required for it to make an action habit?  Are recognition habits sufficient if combined together in to new recognition habit to provide all action habits?

Why do I feel the need for RSR’s?  What was the problem they were going to solve.

In a recognition habit after S1 has occurred and we are conscious of it and we delegate the R and S2 to the subconscious, success comes in the subtle form of the expectation of S2 being matched with the actual occurrence of S2 if we are practicing consciously. But the subconscious does not deal with expectations; it just matches actual S2 with what it should get next after doing R. It did not expect S2 before doing R. If subconscious does not get S2 it alerts consciousness of the current S it got instead and stops doing habit. Effectively the memory trace is still S1 R S2. (subconscious also interrupted by rewarding stimulus (or painful one)) But if subconscious does get S2 and there is an R2 S3 immediately after the S2 in memory and it has been flagged as learnt (what form does this take – a flag on the S2 indicating the expectation of S3 was matched by real S3 when practicing!) then the subconscious can carry on doing R2 and S3. However we have it collapsing the S1 R S2 combination into an S10 when S2 comes in and matches expected S2. Then when next S1 is recognized we start the S10 subconsciously to do R and S2. This should remain subconscious if S10 is first stimulus of another collapsed combination such as S10 R2 S3. It becomes conscious when the next S is not what is being executed. When it becomes conscious then memory is used to investigate the S3 and any known / experienced things to do or S’s to expect.

We introduce new R’s when we are bored (but subconscious habits are by definition boring and there is a conflict with making something a habit and recognizing the boring situation) Maybe only S’s can become boring. As soon as an R is involved there is no boredom. I think novelty cannot be used as a reward for action / R’s that result in novel S’s because novelty only occurs once for each stimulus. I think R’s need a reward that can be a special stimulus that is rewarding each time it occurs. It then gets attached to recognition habits and can be expected. But its rewarding aspect also must decay if obtained too frequently rather like novelty does but just not as quickly.

It’s the SRS’s we flag mentally with success that becomes habits.

S1 new S2 new

S1 (same start habit (no action to do) expect S2) S2 (same and meets expected form habit S10)

S1 (do S10) S2 (S10 recognized)

Action gets put in when two S1’s (or S10’s) in a row.

S1 new

S1 same I didn’t do anything- environment repetitive / boring so do action

R1 picked at random results in K1 (Kinesthetic S) new

S2 new

Later

S1 (expect S1 if do nothing, expect K1 if do R1, K1 was novel rewarding so do it)

R1 done  - > receive K1 same as expected so form habit S20.

Later

S1 do R1 K1 recognized (but doing subconsciously) recognize an S20 same.

Will only do another action if S20 followed by another S20.

20th June 2002

I had a rather interesting experience this morning in my hotel room that may shed some light on how our habits function. I was doing something and had the thought that the newspaper is probably hanging on the door and I started to get up to go and get it. On the way from the bed I got interrupted by something and found myself on the way to the door actually not even yet at the bathroom and had forgotten why I was headed in that direction. I stopped to try to think about what the goal of my walking was and couldn’t. The strategy came to mind to return to the original instigating position to see if the idea would resurface. It actually did before I got back and I went to the door and got the newspaper.

The start of this episode is the most interesting because I had a goal (desirable) I started a habit to perform the next step but to complete the task I needed concentration (conscious support / thinking) during its performance for its success. It had not been practiced to the state where all the steps of going to the door could be done without some thinking involved. My thinking got distracted and I could not pull out of short-term memory what the original idea was but I still knew I was doing something desirable.

The original idea of getting the newspaper was triggered by the appearance of the bed in the morning sun because that is where I lay the paper out to read it on previous mornings.

Obvious the interrupt on the way to door was such that its feeling or duration or content was sufficient to overpower the trace in short-term memory of the newspaper idea. Also there were no clues during the execution of the task which would help remind me on the way as to what I was doing here. In fact at the time I wondered what I was doing I noticed the bathroom as being the next major / closest thing and thought of it but no associated idea that was desirable came to mind, so I said to myself the goal had nothing to do with the bathroom.

Therefore, I deduce that often when completing a task requiring several conscious decisions / interruptions for its completion it helps if the situation around you can be used to remind you of why you are here and what stage of the task you are in. Especially if your thinking is likely to be interrupted by other tasks or events unrelated to the original idea / task.

7th July 2002

I have been reading Habits Their Making and Unmaking by Knight Dunlap and these thought have come to me.

We learn from our mistakes! Practice makes perfect.

Thinking involves the transfer of desirous ends (stimuli / goals) from one thing to another. In order to please someone I now have a goal of doing something. That goal gets transferred into a goal to find something I can use. That gets transferred to a goal of getting on the Internet etc. The motive to get on the Internet is to please someone.

The response to be learned, in short, is an ideal, which is present in the learning, and which is actualized in the result.

If we start with no responses, we have nothing to modify. Therefore, there must be, in the animal, certain initial equipment of unlearned responses.

I need to describe the different types of things remembered. There are skills / habits, events, and ideas/concepts. Maybe ideas and concepts have to form in STM before they can be remembered.

We need to learn the word to skill relationship before instruction can be used to modify our behaviour / skills.

Fear is more than just the expectation of something bad but also the fact that there is nothing known to be done to avoid it. I.e. it seems to happen independently of any actions taken.

Anger originates when the threat comes from the action or expected action of another person or from some object or source that is personified. This then leads to the thought of resisting the action.

Other emotions – rage, jealousy, shame, pride, grief, glad, happy sorrow, pity, pleasure, pain, excitement, strain, disgust, terror, courage, joy, loving, hating

Sensory input that should be considered include hunger – lack of fuel and fatigue / tired – muscles need repair / regeneration / reenergizing.

Moods – apathy, callousness

It would help to sort out feeling / emotions to relate the relevant feelings to the stages of the thinking involved it accomplishing a goal. There is the expectation, desire, followed by the feeling of success or failure. These should be relate-able to happy or sad. Fear related to expectation of failure etc.

“thinking is identical with the thoughts which are its parts”  Aristotle in his de Anima (On the Soul)

“in the history of the individual, knowledge comes before its employment or exercise.”

He states that the sense of touch is had by all animals. But not all animals have the rest of the senses.

“To the thinking soul images serve as if they were contents of perception”

“Hence (1) no one can learn or understand anything in the absence of sense, and (when the mind is actively aware of anything it is necessarily aware of it along with an image; for images are like sensuous contents except in that they contain no matter.”

9th July 2002

The expectations come into the SRS in the following sequence:

Context and triggering Stimulus which matches memory

Idea of resulting stimulus recalled from memory along with the idea of the feeling achieved.

Do the habit (presuming above idea of feeling is good) – responses and stimuli until

Get the resulting stimulus – end of habit execution.

Get the feeling of success as a result of getting resulting stimulus and/or get the

Feeling of good inherent in the resulting stimulus (maybe novelty) (if any, familiarity not being a good feeling but the above success feeling is)

In summary If you want the result (have the idea of the result and a positive feeling) then do the response because it was successful in your past experience.

Thinking could be a learnt process in which the mechanisms of recall, doing, get next (a particular feature from a particular sense or next associated idea or next associated feeling), get what was previous etc.  are the responses and the stimuli are the events detectable in short term memory such as same as last, novel, have habit with success, have a particular feeling etc. The thinking though would start out with the simple means-end goal oriented thinking pattern contained in the above SRS sequence.

Sensory signals = stimuli = sensory input, actions = responses = motor output.

And then there must be a thinking response that given a particular Stimulus will retrieve a desired feature for recall.

Why - Thinking and learning serve the purpose of (are mechanisms for discovering) “what to do next”

I need to address the ability to recognize a subset of the features / stimuli in a series of stimuli which are significant. I.e. the occurrence of b and c is important and should be reacted to whether it occurs as part of; a, b, c, d, e or m, b, n, c   or  x, a, c, d, b.  If all features are available simultaneously then I am sure there exists some pattern recognition algorithm to identify repeating groups of features. If the features are sequentially experienced then some form of STM and associated mechanisms would do the job. This capability is central to generalization!

10th July 2002

I shall use positive as the adjective to describe any feeling that is associated with good or the goal of the agent. I will use negative as the adjective to describe any feeling that is associated with bad or the goal the agent is trying to avoid.

Distress is the feeling from failure to achieve a goal. It’s negative. Pleasure is positive when it results from the success of fulfilling a goal.

I’ve got responses correlated with outputs / actions but paying attention is also something we do. So selecting certain features from the input is something we do. Selection is a response. But it is output to the sensing device to only return certain features and ignore others. It results from the doing of recognition, purposefully doing the paying of attention in order to recognize a stimulus pattern as opposed to reflexively doing it to recognize something. It becomes overt / observable when we move to obtain the input.

 Both must have a goal in mind. The obtaining of the information (a value) for the feature desired. You do the same on memory when you purposefully seek a particular associated feature from experience. Expectations (Ideas) when executed are looking for the similarly featured ideas from memory.

Use the term composite stimulus to refer to a pattern of features as a new stimulus.

11th July 2002

While reviewing the code that cleans up the habit stack it occurred to me that:

Any habit not progressed – failed to get next expected stimulus then they are dropped. – OK for pure stimulus recognition habits (started automatically – and finished automatically) but not for ones started consciously due to desire to accomplish (they most likely contain a response – may not if it is just conscious / concentrated on observation) but are most likely goal oriented response ones. So need to distinguish between consciously and unconsciously started habits – unconsciously started ones must be pretty reliable! The context / trigger must be exact and success always works / accomplished.

11th July 2002                     from note book

Clean up habit stack. Any habit not progressed – failed to get next expected stimulus then they are dropped – OK? For pure stimulus recognition habits (started automatically and finish automatically) but not for ones started consciously due to desire to accomplish (may contain responses – may not if it’s just concentrated observation) most likely are goal oriented response ones.

Need to distinguish consciously and unconsciously started habits. Unconsciously started ones must be pretty reliable! The context / trigger must be exact and success always works / accomplished.

13th July 2002

I have introduced a new bug in the habit recognition software by reversing the use of a stimulus memory match. It was that a stimulus match was first checked as a possible resulting stimulus and then it was matched with any next expected triggering stimulus needed by habits. When I reversed this order of use of a stimulus memory match it no longer worked correctly. So in the series a, b, c: b was first checked as a trigger for b, c before it was checked as a result for a, b and this didn’t work well.

A habit that includes a response must have been started consciously or if it is a sub task of a bigger task then the bigger one must have been started consciously. E.g. you scratch an itch subconsciously but you started it consciously. But you turned the left turn indicator on subconsciously but the bigger habit of driving was started consciously.

13th Aug, 2002             Binon = Binary Neuron

I tried to fix the habit stimulus recognition algorithm by matching the next expected stimulus at the bottom of the tree and matching the next expected habit at the top, but it didn’t match any next expected habits between the top and bottom. A’s stimuli are received this activation propagates up the tree of binons. In real neurons if a stimulus is delayed long enough it will not cause propagation up the tree and the habit will die – the stimuli – recognized and ignored.

16th Aug 2002             Context

Struggling with context – on pg 57 (4th July 1999). The A 12 was the result because the 12 was unfamiliar and had context of an A and attracted conscious attention 1st. But what if a 12 was familiar? It would be on stimulus list and the 1 D would attract attention 1st because it is an unexpected result after having recognized the 1.

16th Aug 2002

If a habit is active (being done) and it gets unexpected next stimulus independent of whether it was started consciously or not, it fails and this attracts attention after any unfamiliar stimuli but before any familiar stimuli because the sequence is new even though the next stimulus is familiar.

Inserted note: If habits I’m doing and it gets – fail unexpected then surprise! – pay attention to same sense – new input. If habit being done subconsciously and it fails then pay attention too. Implies failed habit is put in LTM – conflict with context idea – needs resolving

What is the pleasure in listening to a tune you already know?

19th Aug 2002

I realized SURPRISE takes two forms. Totally novel stimuli or habits (novel pairs of familiar stimuli) cause real surprise. A habit that fails but the next stimulus is familiar (in a familiar context) is also surprise – but surprise of expectation (not expected) and this usually is the source of humour. We laugh at such familiar but unexpected stimuli.

20th Oct 2002

I attended McDonnell Carleton Conference on Neuroscience and Philosophy and started drawing the following diagrams to show progression of binon’s state.

Each binon X has the structure where S1 and S2 are input stimuli which occur in the order S1 before S2.

Each could be an S3 from a hierarchically lower binon in the network. S3 is the stimulus the binon produces when it recognizes S1 and S2.

R1 is the response – action (if any) the binon produces between S1 and S2 occurring. X is the state the binon is in.

Scenario 1 Initially the binon is created when 2 stimuli occur in an order never before recognized. So you get

A new binon exists in the start state S.

Scenario 2 Now 1 occurs again followed by 2.

2 was expected and when occurred success achieved and new stimulus generated saying I recognize 1 and 2 and this is the second time it has occurred so its reliability is pretty high. In essence 3 represents a 1 then 2.

By forming whole hierarchies of these and complicated stimulus sequences can be recognized.  If 1 and 2 occur simultaneously (within the temporal acuity of the sensors) we could recognize combinations of stimuli at a time. The simultaneous stimuli could be ordered – alphabetically numerically or whatever ordering sequence and recognition could be done.

Thought 1 Thought begins in the second scenario when 1 occurs an expectation of 2 occurs before 2 actually arrives. This expectation is the idea of 2.

Scenario 3 Now 1 occurs again but the binon says it expects 2 and will generate 3 but Adaptron knows this already so it is bored and decides to see what the world will do if it perturbs it by generating a response (a random output).

It gets 3 in as a resulting stimulus [instead of 2].

This is the same as scenario 1 where it was lazy and did nothing. At this point it has changed from a stimulus recognizing binon that just has the habit of recognizing S1 and S2 to one that is a response generating binon.

Note Scenario 1 is the reflexive stimulus attracts attention mode but in Scenario 2 it is active – doing paying attention to the sense / feature where it expects 2 to come from.

In Scenario 3 it’s bored with 1 input and reflexively generates 2 response and is no longer expecting scenario 2 stimulus 2. It is all ears – not paying attention to any sense / feature. In Scenario 3 a new binon has been created because the old one (binon M (scenario 1 and 2) is still assumed to work if 1 occurs and nothing is done.

Scenario 4 occurs again and scenario 1 and 2 binon will work but we don’t have proof that scenario 3 binon N is reliable, so we do it rather than scenario 1 and 2 binon M. By do it I now mean N becomes active and does R2 and expects S3 on the appropriate sense / feature. This expectation is the idea of the goal stimulus / resulting stimulus.

If then 3 occurs consistent with expected 3 then binon N produces stimulus 4 saying success I have recognized (and performed) this habit.

23rd Oct 2002

Does not get 2, the binon becomes inactive. 1 occurs again and it is expecting 2 even if 2 is familiar if it occurs binon outputs and this is new novel stimulus 3. 1 occurring could have started several binons active all expecting their next stimulus.

1 occurring could also have been the expected stimulus (2nd one) of several other binons – they will act according to their state.

27th Oct 2002                           Recognition

But now what after habitualized A results in output of a and kinaesthetic input of a. Next A could:

1/ output a again

2/ output something different

3/ output nothing and expect the B.

                                   Thoughts

Ideas are memorized thoughts. Thoughts take place in our minds and most of them we don’t remember but when we do they are ideas. Do we need language to represent these ideas – gives them substance – grounds them to some stimulus (the word)?

22nd Nov, 2002 From Oct 17th – 20th Conference

A recall that results in no match from memory is a recognizable stimulus which we use in thinking.

We have a readiness for a result when we act – if it happens as expected it is confirmation – if it does happen as expected or we don’t have an expectation / readiness then the result is surprise. Expectation is an anticipation (especially if a response is included).

7th Dec. 2002                             Orienting Response

In learning how to perform orienting responses (paying attention responses) the only reward for correct actions is novelty – so this must be used as the feeling of success for learning such responses. Reflexive responses to start with, automatic / reflexive responses to changes in stimuli which then get learned as actions that can result in novel stimuli.