2007

2nd January 2007

UML Class Diagram of senses and sensors

UML Class Diagram of Stimuli and Responses

3rd January 2007                 A and S-Habits

The following are reworded definitions for A and S-Habits that use the terms “expected” or “goal” stimulus for the second stimulus.

Recognition Habit (R-Habit): An R-Habit is implemented as a Binon which receives two input stimuli and produces one output / resulting stimulus. The resulting stimulus signals the recognition of something. There are two types of R-Habits, Sequential (S-Habits) and Parallel (P-Habits). Note that either S or P Recognition Habits may produce the input stimuli.

Sequential Habit (S-Habit): An S-Habit is a Recognition Habit that is triggered by an initial stimulus and then is expecting a second goal stimulus. If the expected stimulus occurs immediately after the initial one then the S-Habit has recognized the sequence of stimuli and it produces its output / resulting stimulus. If the expected second stimulus does not occur next then the S-Habit fails and does not produce its output / resulting stimulus.

Action Habit (A-Habit): An A-Habit is implemented by a Binon and functions similarly to an S-Habit however it generates a response after the trigger stimulus. It only works sequentially. After the initial trigger stimulus occurs the A-Habit produces the output / response and then is expecting its second goal stimulus (S-R-S). If the expected Stimulus occurs the A-Habit has been learnt or repeated else it has failed and some other A-Habit has been formed. When an A-Habit is performed, is its goal stimulus feeling produced?

7th Jan 2007                         Feelings and the experienced stimulus

Question: Is a new sequential habit novel/interesting even if the second stimulus is familiar?

Or what about: Is a new sequential habit novel/interesting even if the second stimulus is boring/uninteresting?  Or does a sequential habit take on the feeling of the second stimulus whenever the sequence first occurs? 

Then when the first stimulus occurs for the second time and the sequential habit is recalled is the interest in performing the sequence based on the sequential habit’s feeling recorded at its first occurrence or is it based on the second stimulus’s current interest level, which may have changed since the sequential habits first occurrence? This interest in performing will determine whether to repeat it or perform a reflexive action instead.

Does the permanence of the stimuli have a bearing on this?

Obviously the questions about the new sequential habit’s feeling is only relevant if the first stimulus is non-permanent because if its permanent you are not going to do any action after it occurs, you are going to perform the S-habit and wait for the second stimulus. If the second stimulus reoccurs you will have recognized the sequence. What will this new S-habit's feeling be?  The first stimulus’s feeling will remain neutral because it is permanent and whatever the S-habits feeling is you will never perform any action if its first stimulus reoccurs.

Let’s assume maximum real-time dynamics such that the feeling of an S-Habit is based on the currently expected feeling of the second stimulus (not on the past S-Habit experience). The S-habit is only contributing the expected second stimulus, not its feeling.

Using:
A and E          are permanent - neutral
B and F          are novel / interesting – never occurred before
C and G         are familiar and neutral, have occurred before
D and H         are uninteresting / boring

B then F:        B occurs, its feeling is novel, attention is attracted to it, and it’s stored in LTM waiting for the second stimulus. B’s feeling is now neutral because it is now familiar. A repeater habit is started expecting B at neutral level. The interest level in a started habit is the expected feeling of the second stimulus at the time it is started. F occurs, its feeling is novel, and obviously it was not expected because B is novel, attention is attracted to F, and it’s stored in LTM waiting for next stimulus. B’s feeling is changed to novel because that was the feeling experienced when F occurred. F’s feeling is now neutral because it is familiar. The S-Habit B,F now exists but has not produced its resulting stimulus so its feeling is novel.

B then G:       B occurs, its feeling is novel, attention is attracted to it, and it’s stored in LTM waiting for the second stimulus. B’s feeling is now neutral because it is now familiar. A repeater habit is started expecting B at neutral level. G occurs, its feeling is neutral, obviously it was not expected because B is novel, provided attention is attracted to G, it’s most recent interesting experience is started or a reflex action is done and thus is stored in LTM waiting for next stimulus. The interest level in a started habit is the expected feeling of the second stimulus at the time it is started. B’s feeling is changed to neutral because that was the feeling experienced when G occurred. G’s feeling is now uninteresting because it has occurred a second time. The S-Habit B,G now exists but has not produced its resulting stimulus so its feeling is novel.

B then H:       B occurs, its feeling is novel, attention is attracted to it, and it is stored in LTM waiting for the second stimulus. B’s feeling is now neutral because it is now familiar. A repeater habit is started expecting B at neutral level. H occurs, its feeling is uninteresting, obviously it was not expected because B is novel, provided attention is attracted to H, it’s most recent interesting experience is started or a reflex action is done and thus is stored in LTM waiting for next stimulus. The interest level in a started habit is the expected feeling of the second stimulus at the time it is started. B’s feeling is changed to uninteresting because that was the feeling experienced when H occurred. H’s feeling continues to be uninteresting. The S-Habit B,H now exists but has not produced its resulting stimulus so its feeling is novel.

C then F:       C occurs, its feeling is neutral, and it is familiar. Assuming it is attended to it will have been stored in LTM. C’s feeling is now uninteresting because it has occurred a second time. There maybe one or more S-habits or A-habits, maybe reflexive that have been started based on C as the trigger stimulus. (or continuing because they expected C? ) A repeater habit is started expecting C at the uninteresting level. F occurs, its feeling is novel, obviously it was not expected by any habits, all habits fail, attention is attracted to F, and it’s stored in LTM waiting for next stimulus. C’s feeling is changed to novel because that was the feeling experienced when F occurred. F’s feeling is now neutral because it is familiar. The S-Habit C,F now exists but has not produced its resulting stimulus so its feeling is novel.

C then G:       C occurs, its feeling is neutral, and it’s familiar. Assuming it is attended to it will have been stored in LTM. C’s feeling is now uninteresting because it has occurred a second time. There maybe one or more S-habits or A-habits, maybe reflexive that have been started based on C as the trigger stimulus. (or continuing because they expected C? ) A repeater habit is started expecting C at the uninteresting level. G occurs, its feeling is neutral, and it may have been expected. If G was expected the habit is recognized ( or is continuing? ), the previous C,G S or A-habit is marked ignore and this new one stored.  If G was not expected all the habits fail. Attention is attracted to G, it’s most recent interesting experience is started or a reflex action is done and thus is stored in LTM waiting for next stimulus. C’s feeling is changed to neutral because that was the feeling experienced when G occurred. G’s feeling is now uninteresting because it has occurred a second time. The S-Habit C,G now exists but has not produced its resulting stimulus so its feeling is novel.

C then H:       C occurs, its feeling is neutral, and it is familiar. Assuming it is attended to it will have been stored in LTM. C’s feeling is now uninteresting because it has occurred a second time. There maybe one or more S-habits or A-habits, maybe reflexive that have been started based on C as the trigger stimulus. (or continuing because they expected C? ) A repeater habit is started expecting C at the uninteresting level. H occurs, its feeling is uninteresting, and it may have been expected. If H was expected the habit is recognized (or is continuing?), the previous C,H S or A-habit is marked ignore and this new one stored.  If it was not expected all the habits fail. Attention is attracted to H, it’s most recent interesting experience is started or a reflex action is done and thus is stored in LTM waiting for next stimulus. C’s feeling is changed to uninteresting because that was the feeling experienced when H occurred. H’s feeling continues to be uninteresting. The S-Habit C,H now exists but has not produced its resulting stimulus so its feeling is novel.

D then F:       Same as C then F with D becoming novel.
D then G:       Same as C then G with D becoming neutral.
D then H:       Same as C then H with D becoming uninteresting.

11th Jan 2007                      Permanent Stimuli, A-Habit Feelings

I believe permanent stimuli should behave just like the non-permanent ones as far as feelings are concerned. They should not stay neutral but be interesting or uninteresting also.

An Action Habit has no feeling itself because it is never collapsed. The interest in repeating it is based on the current feeling of its goal stimulus. This means that as soon as Action Habits are stored in memory they can be marked with the Ignore feeling. But I mark them with interesting so they appear unlearnt then mark them Ignore once repeated / learnt.

18th Jan 2007                      Permanent Stimuli

A possible strategy is to not allow a Primitive stimulus or S-Habit to become permanent until all possible responses have been tried and no uninteresting expectations have been received. You would retry all the possible responses again. Why you might do this I don’t know. Maybe it’s a solution looking for a problem!

20th Jan 2007                      Classical Conditioning – Pavlov

Classical conditioning implies that the trigger stimulus of an S-Habit becomes the trigger for the response that is already produced by the second stimulus of the S-Habit. Thus the S-Habit's 1st stimulus ends up prematurely triggering the response.  I suggest that what really is happening is that the 1st stimulus is causing an expectation of the second stimulus and this expectation is triggering the response just as though the actual 2nd stimulus would. This is what happens in thinking. The expectation of a stimulus, i.e. the thought of a stimulus often triggers the start of the action.

22nd Jan 2007                     Learning does not need repetition

Adaptron does not need to repeat a sequence to have learnt it. As long it has occurred once it is known and therefore learnt and is expected to always give the same expected 2nd stimulus based on an assumption the world/environment is deterministic. This means that the sequence documented on the 8th March 2006 should actually be:
Input:              A B A
Output:                  a

23rd January 2007              Interest in the Trigger – A-Habits

When an A-Habit is done there is an expected interest level of the goal stimulus.  Read 17th March 2006. If the goal stimulus turns out more interesting than expected the trigger stimulus becomes more interesting. If the goal stimulus turns out less interesting than expected the trigger stimulus becomes less interesting. A similar transfer of interest does not appear to occur or work in an S-Habit. In an S-habit if the trigger stimulus occurs a second time, then we know exactly what to expect (independent of how interesting it is) and perform a reflexive output. This places the interest level in exploration higher than re-experiencing interesting second stimuli.

But isn’t doing nothing = listening the same as doing something. If AB has occurred once and B in interesting then when A occurs a second time shouldn’t we repeat the listen, expecting an interesting B? Then when the interesting B occurs the actual feeling matches the expected feeling – no change in the interest level and it is not interesting so A becomes neutral. Then B becomes neutral because it has reoccurred.

But how to handle the situation when trigger and / or second stimuli turns permanent? Do they still have an interest level?

The level of interest is a graduated stimulus. And it is the change in the level of interest that is rewarding or punishing as far as whether the trigger stimulus is worth obtaining. I.e. the change in the level of interest determines the level of interest of the trigger stimulus.

So the following table provides the trigger stimulus’s new interest level:

Expected                   Experienced             Difference     Trigger gets (threshold truncated)
Interesting                 Interesting                   0                 Neutral          - habituation
Interesting                 Neutral                       -1                 Uninteresting - let down
Interesting                 Uninteresting             -2                 Uninteresting
Neutral                       Interesting                  1                 Interesting
Neutral                      Neutral                        0                 Neutral
Neutral                      Uninteresting             -1                 Uninteresting
Uninteresting             Interesting                  2                 Interesting
Uninteresting             Neutral                       1                 Interesting     - improvement
Uninteresting             Uninteresting              0                 Neutral         - habituation

24th January 2007              Permanent Stimuli

The objective when a stimulus has turned permanent is to recognize any sequence that may follow the permanent stimulus. If the permanent stimulus is A then whenever A occurs we want to start all known S-habits triggered by A to be started and expecting their results.

  • If B is then experienced we could have all the habits fail so store 'A' in LTM. The 'A B' sequence is a new one. Maybe should always store B as well.
  • If B is not permanent it will be stored in LTM either as the trigger of a habit or because it was novel or because it’s being followed by a reflexive response.
  • If B is permanent and all the 'A' habits fail then do we start all B S-Habits? No we want to capture this unexpected sequence, so store B in LTM and start no habits.

However, if B occurs and it matches the expected stimulus of an 'A' S-Habit, that habit sequence is recognized. The 'A B' sequence has now reoccurred. Should the A B S-habit be collapsed / recognized?

  • If B is not permanent it will be stored in LTM either as the trigger of a habit or because it was novel or because it’s being followed by a reflexive response.
  • If B is permanent then the AB S-Habit should be recognized.

25th January 2007              Which A-habit to do

Now that an A-habit is learnt the 1st time it occurs (it is not repeated to learn it, the environment is assumed to be deterministic and is expected to produce the same second stimulus for the given response whenever we do it again in the same situation) we may have an AaC A-habit and an AbB A-habit that have been done. When the next A occurs which of the two A-habits will get performed or neither and why? The two matches will have an interest level based on the current expected interest of their goal stimuli. Recency of the two A-habits should play no role in the decision unless both A-Habits have an equally interesting goal. So the highest interest A-Habit should be chosen and the most recent if they are of equal interest. But then it will only be done if it has an interest level higher than the current concentration level because a continuing habit should continue unless something else is more interesting / worth doing. But if the most recent A-Habit is expected to be boring (i.e. get an uninteresting second stimulus) will you have any interest in doing an earlier A-habit which expects an interesting goal or will you try the next reflexive action?

Maybe we go back and do any and all A-habits that expect an interesting goal until there are none that expect something interesting at which point we do a new reflexive response. When all reflexive responses have been tried and no past A-Habit expects an interesting response then the trigger stimulus becomes permanent.

31st January 2007               Interest in past experiences

If we are deciding whether to repeat a past A-Habit the rules on 25th January seem correct. Except when the most recent one that is not an Ignore is uninteresting we don’t look back any further in past experiences. We do a reflexive response. If the most recent one is neutral or interesting then we look backwards through past experiences for the most interesting one. Do we stop at one that is uninteresting and not go back any further in memory?

This kind of question is very similar to the fear of snakes. Initially you might have some good experiences with snakes based on “all sorts of actions” done with/to them. Then one bites you when you perform action X. You now have a fear of snakes in general. You don’t associate the bad experience with the action X in the presence of a snake. Now you don’t do any actions which will get you close enough to repeat any of the “all sorts of actions”. But what if, unexpectedly you are placed with a snake? Do you freeze? That would be do nothing, just observe. Or do you perform some type of reflexive action? E.g. turn around and run? Or do you keep your cool and remember the one action X that got you bitten and don’t repeat that. However you do remember the “all sorts of actions” and pick an appropriate one?  What if you are cured of your snake phobia by gradually replacing the fear of snakes with a likeness for snakes and then you get close to a snake. Surely you wouldn’t repeat action X but might do “all sorts of actions”?

Read test run 2 from today about storing permanent trigger stimuli in memory even if we don’t know whether the next stimulus is permanent or not. This won’t work because the habit that has its context finally stored is not known until it gets its unexpected stimulus.

12th February 2007             Interest in S-Habits

With the new interest propagation scheme documented on January 23rd, such that the trigger stimulus gains the interest of the goal stimulus working backwards through a habit we end up with an infinite loop of S-Habits always interested in the next stimulus when we repeat ABC in a loop. Maybe a different interest propagation scheme works for S-Habits than A-Habits. No action has been done so maybe the interest in the second stimulus should not be given to the first stimulus. If the second stimulus of an S-Habit is interesting should it affect the interest in the first stimulus? On the 23rd Jan, I wrote “In an S-habit if the trigger stimulus occurs a second time then we know exactly what to expect (independent of how interesting it is) and perform a reflexive output.”

However, with an S-Habit we still have an expected second stimulus and we have an experienced second stimulus. If the trigger is not affected we still need to modify the second’s feeling due to repetition. An expected second stimulus cannot be novel because it has been experienced before.

The difference with an S-Habit is that the sequence gains the interest from the second stimulus, not the trigger stimulus. Then when the trigger stimulus re-occurs, it is the interest in the sequence being repeated that makes us wait and listen or react.

14th & 15th January, 2007             Attention

After Adaptron has received all of its internal and external primitive stimuli, it does the P-Habit matching and then the S-Habit matching. Then it has a list of stimuli comprised of primitive stimuli, P-Stimuli and S-Stimuli. Plus it has some S-Habits that are continuing and some that have failed. It also has some A-habits that have failed, completed or are continuing. It then goes about determining which attracts attention. Once this has been done it will do one of:

  1. Novel reaction, stop and store stimulus,
  2. Start a habit based on attended to stimulus, may result in reflexive response
  3. Reflexive response,
  4. Continue at current concentration.

The things that can attract attention are:

  1. A LTM unexpected stimulus (novel stimulus)
  2. An unexpected S-Stimulus (that fails to get any of its expected next stimuli)
  3. An unexpected primitive or P-stimulus (different based on sensor or P-Habit)
  4. A pleasant stimulus
  5. A unpleasant stimulus
  6. An interesting stimulus
  7. A boring / uninteresting stimulus
  8. An S-Stimulus that has a context of permanent stimuli
  9. An A-Habit that fails to get any of its expected goal stimuli (unexpected goal)
  10.  An A-Habit that completes
  11.  A neutral primitive stimulus, P-Stimulus or S-Stimulus

But all these must have an interest level greater than the current concentration to attract attention.

1 will obviously cause 2 and 3 as well. 2 will cause 3. A stimulus that does not cause 3 must be the same as the last one from that sensor and therefore must complete a repeater S-habit for that sensor.

Let’s leave out pleasant and unpleasant stimuli at this point and match up what to do based on whether the stimulus attracts attention.

  A. Stop & Store B. Start Habit C. Reflexive Response D. Continue
1 LTM Unexpected Novel one      
2 S-Habit Unexpected Context Unexpected as trigger    
3 Sensor Unexpected        
6 Interesting        
7 Boring context   Yes  
8 Context S-Habit context S-Stimulus as trigger    
9 A-Habit Goal Unexpected   Unexpected as trigger    
10 A-Habit gets Goal   Goal as trigger    
11a Neutral Stimulus – nothing continuing   Based on most interesting experience    
11b Continuing S-Habit   Alternates with context   Yes

Pseudo-code may do a better job of describing what to do:

  • If LTM Novel - largest novel P-stimulus then
    • Some habits will fail. The novel stimulus will have an interest level based on its amount of change / difference from sensory repeater habit.
    • If the current conscious habit fails then
      • Concentration reduced to neutral and attention to the novel stimulus. Store any context of conscious S-Habit in progress and update their learnt status?, store novel (unexpected) one (can’t start any new Habits based on novel stimulus, no response, nothing to continue, repeaters completed) and start sensory expectation of a repeat. Stimulus has lost its novelty / interest.
    • Else the current conscious habit gets its next expected stimulus, completes or is continuing then
      • If the novel stimulus is a higher concentration level then
        • It interrupts the current habit and we do same as above.
      • Else the novel stimulus is ignored
  • Else if all S-Habits get unexpected then             `else it’s familiar.
    • Store any context of S-Habits in progress and update their learnt status?, start the most interesting habit based on the unexpected stimulus as the trigger. Start sensory expectations of a repeat. It’s a new sequence.
  • Else if boring non-permanent stimulus then
    • Store any context of S-habits in progress and update their learnt status?, store boring one, update learnt status of any that complete, and do reflexive response.
    • It’s not boring, could be permanent.
  • Else if an A-Habit (conscious one) gets unexpected goal then
    • Update the learnt status of the A-habit and start the most interesting habit based on the unexpected stimulus as the trigger.
  • Else if A-Habit (conscious one) gets expected goal then
  • Else if a permanent stimulus then
  • Else if S-Habit with context completes then
  • Else if sensor detects a difference then
    • The repeater habit for this sense has failed
  • Else if have a continuing habit then
    • Must be recognizing an S-habit comprised of permanent stimuli
  • Else A neutral stimulus with nothing continuing then

There are always S-Habits or A-Habits in progress. Even when a reflexive response has been done the sensors are still expecting the same stimulus as last time. The conscious (current) habit is the one we are doing expecting the most interesting result. Others are executing subconsciously. They may be continuing. They may be expecting repetition and their concentration level is based on the desirability of completing the S-habit sequence which one would think is boring. But for repeaters maybe it is based on the interest in the stimulus that has just occurred. This would mean a repeating S-Habit could be concentrating at neutral or boring.

Unexpected stimuli attract attention because a habit or a group of habits fail. Habit completion recognizes no change or difference.  If there is no change and any stimulus is boring then a reflexive response is done to create change / difference. If there is no change and all stimuli are neutral then are there any experiences that will obtain an interesting goal? If so do them. If experience says no interesting goal possible then a reflexive response is done to create change / difference. If experience says to expect a boring goal then a reflexive response is done to create change / difference.

So, multiple S and A-habits are executing. One is the conscious habit with highest interest level. The next stimuli occur and the habits are done. Any habit that fails makes its actual stimulus available for attention.

  • If the conscious habit fails then concentration reduced to neutral.
    • The unexpected stimulus with the highest interest above the concentration level attracts attention.
    • Or if the highest is neutral then a boring unexpected stimulus attracts attention.
    • Or they are all neutral and the one that caused the conscious habit to fail is attended to.
  • If conscious habit continues then
    • An unexpected stimulus with a higher interest than the concentration level attracts attention so it does not continue.
    • Or they are all of less interest than the conscious one and are ignored (boring ones too)
  • If conscious habit completes then
    • An unexpected stimulus with a higher interest than the concentration level attracts attention so it does not complete.
    • Or they are all ignored (boring ones too) and the goal or result is attended to.
  • No habit fails but zero or more are continuing and zero or more complete then
    • Go through all stimuli looking for any that might have a past experience with a higher expectation of interest than the current concentration level and start it / them.

16th February, 2007            Permanent Stimuli feelings

If a permanent stimulus has a relevant interest then there is a desire to obtain it or avoid it. This should not be happening. When it is permanent it is not to be responded to, just recognized as part of a larger S-Habit. So, all permanent stimuli should be given a neutral feeling / interest level. There should be no interest in pursuing or avoiding permanent stimuli. Since a permanent stimulus can’t be boring it cannot trigger a reflexive response, which is valid.

17th February, 2007            Boring repetition

Repeater habits are started on all senses and one is started for the last conscious stimulus to detect a change / unexpected stimulus. But if they get the same stimulus repeated it is boring. This is okay if the expected repeating stimulus is a permanent stimulus; the repetition can be ignored. But if the stimulus repeating is not yet permanent it should be boring / uninteresting and this should trigger a reflexive response. The uninteresting stimuli should attract attention before the unexpected ones if both occur at the same time. If we are concentrating on an interesting habit and it is continuing and a boring stimulus occurs this should be stronger than the habits concentration. So feelings of stimuli should dominate over feelings / interest in / concentration of active habits. Interest in active habits is the expectation of the feeling whereas the stimulus feeling is an actual feeling and is higher priority than an expected feeling.

If you experience a pleasant and unpleasant stimulus at the same time of equal potency you are most likely to react to the unpleasant one because that means you will survive for another day in which you can experience the pleasant one without the unpleasant one. The same is true for an interesting and boring stimulus occurring at the same time. The boring one will take priority in attracting your attention.

                                                Distraction

Failed habits are a way of detecting unexpected / changing / novel stimuli. Once these stimuli are identified it is their interest level that needs to be greater than the concentration level for them to attract attention. An interesting stimulus needs to be greater than or equal to the concentration level to distract you. Since concentration level is based on the expectation of interesting goal but an interesting stimulus is an actual goal. Equal interest level to concentration level causing an interrupt is why we can easily get distracted in pursuit of a goal and then get distracted again from the first distraction.

So an actual non-permanent S-Habit stimulus feeling should be checked along with all non-permanent primitive and P-Stimuli feelings for any that are greater than (absolute value wise) or equal to the current concentration level. And the negative / uninteresting stimuli before the interesting ones of equal intensity.

A boring stimulus must be conscious to be boring and there is no less or more boring – it’s not a range like interest level. It only attracts attention if all other stimuli are neutral because anything with a minimal interest level is better than boring at attracting attention.

5th March 2007                    Attention algorithm

The new attention algorithm should do the following after all P and S-Habit matching has been done. All the stimuli should have interest level (IL) feelings (novel if unexpected), boring if consciously repeated and neutral if permanent. The concentration level (CL) should be neutralized if the conscious habit failed or completed. It should remain at its level if the conscious habit is continuing.

  • If CL > 0 then
    • Find stimulus with IL >= CL
    • If none then continue
    • Else start habit based on stimulus found
  • Else (CL = 0)
    • Find any boring stimulus
    • If found do reflexive response
    • Else Find stimulus with IL > 0
      • If found then start habit based on stimulus found
      • Else do reflexive response.

If just the stimulus current IL is used in the above algorithm then it reacts based on no thinking or processing of past experiences, only on the stimulus’s last IL set based on its most recent experience. If experience is used then other past recalled experiences provide other ILs. If thinking is turned on then the other past experiences are explored for possible higher ILs.

6th March, 2007                   Current Attention algorithm

The current algorithm is as follows:

  • Input primitive sensory stimuli
  • Do P-Habit matching
  • Do S-Habit matching
  • Find any continuing S-Habit, continuing A-Habit, recognized/completed habit and completed boring habit.
  • If none continuing nor completed then CL = Neutral
  • If doing S or A-Habits then
    • If completed boring habit then DontDoIt = True, attend to boring stimulus
    • Else if none continuing nor completed then
      • Find the unexpected stimulus for attention
      • Flag it if were trying to recognize sequence
      • Clear consciousness
    • Else if none completed – at least one continuing then no change
    • Else if one completed then
      • Attend to resulting S-Habit stimulus or A-Habit goal stimulus
      • Flag it if were trying to recognize a sequence
      • Clear consciousness
      • Keep habit info.
  • If not yet attending to anything then
    • If any P-stimuli or primitive stimuli are different then attend to the biggest difference.
  • For attend to stimulus or all of them if none to attend to
    • While not got a DoIt nor a DontDoIt
      • If not a permanent stimulus then
        • Find the first boring or highest interesting experience for the stimulus
        • If Find one then
          • Think about it
          • If Wanted or Novel then DoIt = True
          • Else if Feared or Boring then DontDoIt = True
          • Else if thought Wanted or Novel then DoIt = True
      • Else do the permanent stimulus at most recent S-Habit experience
    • End while
  • End loop
  • If nothing to attend to set it to biggest stimulus or what attended to last time around.
  • If attend to stimulus is novel then
    • Store stimulus, stop all S-habits, clear consciousness, start a repeater
  • Else if DontDoIt then
  • Do reflexive response and start a repeater if still not permanent
  • Else if DoIt then
    • If permanent stimulus or IL > CL then
      • Store the context
      • Start the habit based on the experience
      • Start a repeater if not permanent
  • Else if no current habit then
    • If not permanent then
      • Do reflexive response and start a repeater if still not permanent
  • End Doing
  • Start any background S-Habits based on permanent triggers if not stored trigger.
  • Perform response

7th & 8th March, 2007         Stimulus priorities

The current priorities of stimulus formation are:

  1. detect primitive stimuli
  2. form P-Stimuli from above and other P-Stimuli
  3. form permanent S-Stimuli from any permanent above or other permanent S-Stimuli
  4. form S-Stimuli from the above
  5. Listen, execute, perform reflexive action or mark permanent

Rework the attention algorithm:

  • Input primitive sensory stimuli
  • Do P-Habit matching
  • Do S-Habit matching
  • Find any continuing S-Habit, continuing A-Habit, recognized/completed habit and completed boring habit with non-permanent result.
  • If doing S or A-Habits then
    • Else if all fail to get expectation then
      • Find the unexpected stimulus for attention
      • Flag it if were trying to recognize sequence
      • Clear consciousness, CL = Neutral
    • Else if none completed – at least one continuing then no change
    • Else if one completed then
      • If completed boring habit then
        • DontDoIt = True, attend to boring stimulus
      • else
        • Attend to resulting S-Habit stimulus or A-Habit goal stimulus
      • Flag it if were trying to recognize a sequence
      • Clear consciousness
      • Keep habit done info.
  • If not yet attending to anything then
    • If any P-stimuli or primitive stimuli are different then attend to the biggest difference.
  • For attend to stimulus or all of them if none to attend to
    • While not got a DoIt nor a DontDoIt
      • If not a permanent stimulus then
        • Find the first boring or highest interesting experience for the stimulus
        • If Find one then
          • Think about it
          • If Wanted or Novel then DoIt = True
          • Else if Feared or Boring then DontDoIt = True
          • Else if thought Wanted or Novel then DoIt = True
      • Else do the permanent stimulus at most recent S-Habit experience
    • End while
  • End loop
  • If nothing to attend to set it to biggest stimulus or what attended to last time around.
  • If attend to stimulus is novel then
    • Store stimulus, stop all S-habits, clear consciousness, start a repeater
  • Else if DoIt then
    • If permanent stimulus or IL > CL then
      • Store the context
      • Start the habit based on the experience
      • Start a repeater if not permanent
  • Else if DontDoIt then
    • If not permanent then
      • Do reflexive response and start a repeater if still not permanent
  • Else if no current habit then
    • If not permanent then
      • Do reflexive response and start a repeater if still not permanent
  • End Doing
  • Start any background S-Habits based on permanent triggers if not stored trigger.
  • Perform response

I had this test in the loop:

            If attended to stimulus is boring then DontDoIt = True

But then realized this stimulus has already occurred and the boring feeling comes from the conscious habit getting its expected stimulus which was boring not from paying attention to a boring stimulus.

10th March, 2007                 Priority of what to do next

  • Can background recognition habits or repeater habits be done subconsciously
  • Graduated stimuli – levels of interest and concentration – change and absolute values
  • Interest feelings become a source of stimuli
  • Thinking one action ahead – or whole train of thoughts – goal directed
  • Pleasant, unpleasant stimuli and degrees of pleasantness (graduated)
  • 2 or more senses – P-Habits functioning – categorization / generalization / specialization
  • Building long action sequences and reusing them
  • Actions started subconsciously – needs 2 or more senses and / or thinking
  • 2 or more devices – parallel actions

12th March, 2007                 Novelty versus boredom

Which stimulus should attract attention, a boring repeat of a kinaesthetic stimulus versus a novel external stimulus? I think the unexpected external one should. The kinaesthetic stimulus is not determined to be boring until the conscious habit expecting it succeeds, while the different / unexpected external stimulus is determined by the sensor itself when its repeater habit fails. Another way of looking at it is that failures of lower level (closer to sensor) habits attract attention before failure of higher level habits (closer to Sequential Habits or Action Habits).

So I switched around the order of attention attraction as follows:

  • If not yet attending to anything then
    • If any P-stimuli or primitive stimuli are different then attend to the biggest difference.
  • If doing S or A-Habits and not yet attending to anything then
    • Else if all fail to get expectation then
      • Find the unexpected stimulus for attention
      • Flag it if were trying to recognize sequence
      • Clear consciousness, CL = Neutral
    • Else if none completed – at least one continuing then no change
    • Else if one completed then
      • If completed boring habit then
        • DontDoIt = True, attend to boring stimulus
      • else
        • Attend to resulting S-Habit stimulus or A-Habit goal stimulus
      • Flag it if were trying to recognize a sequence
      • Clear consciousness
      • Keep habit done info.

13th March, 2007                 Recognizing stimuli

A pair of permanent stimuli is not recognized as a new sequential stimulus until they have both occurred in a row. However when a stimulus is just becoming permanent, i.e. it is not permanent,  we pay attention to it, go to perform its next reflexive response and find there are no more responses, it is not stored in LTM but started as a sequential recognition habit just in case the next stimulus is permanent too. If the permanent stimulus has a context when it just becomes a permanent stimulus should the context be kept or stored in LTM?  I think the context should be stored.

14th March, 2007                 Graduated environment

I am about to start adding graduated stimuli to Adaptron and realized a simpler environment for testing it (than 1st Sept 2006) is a linear hall with a wall at the end. Adaptron always faces the wall at the end, no rotation. It can move +ve or –ve amounts and measures the distance to the wall with a range finder.

                                                Novelty and Differences

When a novel (unexpected) stimulus occurs, its amount of change determines if it interrupts consciousness based on the current concentration level. But can the change be used as a measure of the novelty of this new stimulus. I don’t think so. The amount of change determines the interest level in the stimulus and the desirability to re-experience it. Its novelty is either yes or no and is greater than any interest level. That is, a novel stimulus takes precedence over any level of interest.

15th March 2007                  P-Habit Production

The idea from the 10th March 2006 to only produce P-Habits where the two source stimuli are different is correct. If we were producing all the higher level P-Habits for numerous sense / features we would be producing an enormous number of highest level P-Habits because we would have one for each time any single sense / feature stimulus was different. In other words we would have a highest level P-Habit for every possible combination of values of stimuli from the sense / features. If this were happening in the human brain we would have a picture perfect memory of every situation we had ever been in. What is happening is the recording of those stimuli combinations that are different from the last sensory experience, those that are different from expectations and those we consciously pay attention to.

In the current version of Adaptron I produce all the higher level P-Habit combinations and mark them “Ignore” so that they will not attract attention but when an active S-Habit needs to check its expected goal stimulus which is a P-Habit it can be found on the S-List quickly. In a future version checking expected goal P-Habit stimuli will have to be done algorithmically because we cannot expect to generate all the P-Habit stimuli and put them on the S-List.

Contradictory to the 3rd March 2006 thinking the production of higher level P-Habits is based on the two source stimuli being different from the expected value on their respective senses. So the production of the Resulting P-Habit is as follows:

1st Stimulus              2nd Stimulus                       Resulting P-Habit Stimulus
Novel & Different      Novel & Different                 Novel one produced
Novel & Different      Familiar & Different              Novel one produced
Novel & Different      Familiar & Same                  New one produced marked Ignore
Familiar & Different  Familiar & Different              Novel one produced or existing one found
Familiar & Different  Familiar & Same                  New one produced or existing one found
                                                                              marked Ignore
Familiar & Same      Familiar & Same                  Existing one found marked Ignore

Note: Novel if both different. Familiar & Different if stimulus has changed from previous one. Resulting Stimuli are not graduated so don’t have a difference. An active S-habit may have an expected P-habit stimulus made up of two stimuli at least one of which didn’t change. In this case the S-Habit will match with the one produced but marked Ignore.

18th March, 2007                 Differences and Novelty

Attention is first attracted to a difference at the sensory level. This level cannot determine Novelty. And with more than one sense it is possible to have a novel stimulus attract attention along with other stimuli and attention is attracted to a combination P-Habit stimulus which includes the novel stimulus. The P-Habit stimulus becomes familiar but the novel stimulus remains novel. This allows for the combination sensory same but LTM novel.

                                                Priority of attention

So a sensory difference attracts attention first. In the case of only one sense a novel stimulus will always produce a difference. If the change / difference in the stimulus is greater than or equal to the concentration level the current conscious habit is interrupted. Then expected stimuli are examined. If all expectations fail then we have a new situation and we pay attention to the stimulus received on the expected sense / feature. If one of the expectations is found then we complete the habit and have recognized the sequence or it is continuing.

                                                Interest Levels

Then each familiar stimulus has an interest level. It is initially determined based on the amount of change that occurred with it when it first occurred. Then its interest level decreases each time it is re-experienced but may change if it is the trigger for an A-habit that gets an interesting goal. This interest level does not play any part in attracting attention but does help determine which habits to perform.

Interest in the whole does not propagate to the interest in its parts and a change in interest in a part does not change your interest in the whole.

20th March, 2007                 Interest Levels of S-Habits & P-Habits

An S-habit comprised of two stimuli is initially novel and given an interest level of 1 (its two source stimuli must be permanent at interest level neutral = 0). The same for a new P-Habit it starts at interest level of 1 independent of the interest level of its two source stimuli. When a new graduated primitive stimulus occurs its initial interest is set to the amount of change that happened at the same time. If it becomes the goal of an A-Habit then the A-Habit’s concentration level will be set to that value. When the A-habit is done the trigger will get its interest level based on the goals interest level versus the concentration level. In this way S-Habits and P-Habits acting as triggers can get interest levels higher than 1. Interest levels in all stimuli are reduced whenever they are consciously added to LTM. They are halved until they reach neutral = 0 and then they become boring = -1.

21st March 2007                  Interest Levels - Concentration

They are testing the fire alarms today in the building. When it originally went off I was quite startled due to the change in volume it caused. After a few seconds I went back to reading while it was going on in the background. It only lasted for a minute. Five minutes later it went on again; I was not startled and mentally acknowledged what it was. I then went back to reading. What level of concentration is required to block out loud background noises? Or is it more correct to assume as long as there is a background habit recognizing the loud noise conscious concentration is not interrupted. This means that A-habits with concentration levels of 1 are interrupted by changes with values greater than or equal to 1 unless there are background habits using up the changes (recognizing the expected stimuli).

I have always liked a quiet environment in which to study. Other people are happy to study with background music. They must have background habits operating to use up the music. Then these habits fail when an unexpected song is played. They probably appreciate the interruption from study to listen to the song.

A possible strategy is to only allow interest levels greater than one for graduated stimuli. Thus P and S-Habit stimuli do not gain interest levels greater than 1 because they are not graduated. Then concentration is never greater than level 1. An S-Habit that expects a goal stimulus with a change amount greater than 1 does not execute at a concentration level greater than 1. It executes at level 1 if we are interested in it and level 0 if it is there just to absorb background stimuli. It is possible that any habit concentrating at level 1 is conscious and at level 0 is an unconscious one. This corresponds to making all permanent stimuli have an interest level of 0. So graduated stimuli on the S-List may have a large change / difference but as long as at least one zero concentration level habit gets it as an expected stimulus then it does not interrupt consciousness.

I have experimented with Adaptron to highlight the S-List stimuli that are used at least by one S-habit and the ones left that are not used by an S-Habit. These should be the ones that attract attention. This is not currently the case. Attention is attracted to the one with the smallest context. If two or more occur with no context or the same context it chooses based on the order in the S-List. This happens when there are two possible interpretations for a sequence of stimuli such as A(BA) or (AB)A.

22nd March, 2007                Attention algorithm

So a new attention algorithm might be something like this:

1.         Perform S & A-Habit matches. Do not pay attention to any stimulus that was expected by a habit. It is “used” and has satisfied a habit that was expecting it. This is independent of its interest level or change / difference. This corresponds to the concept that if all habits fail to get their expected stimulus we must have an interesting unexpected stimulus (it may be novel or familiar).

2. If there are unused ones then

  • From the group with the shortest context
    • If there are ones with any changes / differences then
      • Form a P-Habit stimulus, which will attract attention. Stop doing any conscious habits. We have unexpected stimuli.
    •             Else if concentrating then
      • Continue with conscious habit – assuming it got its expected stimulus.
      • The unused ones are not different enough to attract attention.
    • Else if a repeat occur then
      • We are bored and pay attention to the repeating stimulus and do a reflexive response.
    • Else not concentrating and there are unused ones with no change so
      • Pay attention to the most primitive / smallest first.

3. Else there are no unused ones so

  • If concentrating then
    • Conscious habit succeeded and should continue. If there are any additional expectations these can be added to Habit list. If an A-Habit completed then the goal stimulus has been used. It should pay attention to the goal and update learnt status of A-Habit. If an S-Habit completed then the result should be unused and paid attention to. If the S-habit result was novel we would have had an unused stimulus. If the S or A-Habit failed then we would have had an unused stimulus.
  • Else if a repeat occur then
    • We are bored and pay attention to the repeating stimulus and do a reflexive response.
  • Else not concentrating so
    • A background A-Habit will complete and the goal will be used and only attract attention if it is interesting? A background S-Habit will complete or continue.
    • Pay attention to the most primitive / smallest first with the smallest context.

 If it did not get its expected stimulus then pay attention to stimulus obtained instead.

29th March, 2007                 Differences / Changes – graduated stimuli

The fact that two sequential stimuli on the same sense / feature are different attracts attention, however we must be conscious of them both in order to be conscious of the difference / change in magnitude for graduated stimuli. Thus interest levels are based on the fact that there is a change but not necessarily on the magnitude of the change. Once we are conscious of the magnitude of the change / difference this becomes a stimulus itself. It’s like being conscious of our feelings. It now is recalled and has expectations based on past experiences of the same magnitude of change for this sense / feature. Can a pattern of changes be recognized in another sense / feature? You can immediately recognize “Happy Birthday” in different keys. But can you recognize it in the up and down motion of somebody’s hand who is pressing imaginary notes on a vertical piano keyboard? Can you even recognize it from the hand motions on a piano keyboard while being unable to hear it, assuming you have not seen someone play it before?

Is it possible that neutral interest and boring interest are the same as far as recording this interest level is concerned? They both cause a reflexive response when the past experience is recalled and bored is only determined when the last stimulus is consciously the same as the previous.

The newly minted attention algorithm is as follows:

1. Perform S & A-Habit matches.

2. If there are unused ones then

  • From the group with the shortest context
    • If there are unused ones with any changes / differences then
      • Form a P-Habit stimulus, which will attract attention. Stop doing any conscious habits. We have unexpected stimuli.
    • Else if there are any that are different, match the context but used
      • Pay attention to the most primitive / smallest one
    • Else if not concentrating and there are unused ones with no change then
      • Pay attention to the most primitive / smallest first.
    • Else concentrating but have unused stimuli so
      • Pay attention to same sense / feature as expected.

3. Else there are no unused ones so

  • Pay attention to the most primitive / smallest first with the smallest context.

30th March, 2007                 Generalization & P-Habits

The current version of Adaptron does not appear to be generalizing two stimuli. The rules for recognizing a P-Habit stimulus (result) should be as follows. Current interest levels can be Ignore, Novel, Neutral, or Boring.

1st Stimulus  2nd Stimulus Resulting Stimulus Difference & Interest
Different        Different        Different        &         If it’s just created then it’s Novel
                                           Different        &         If exists use its current interest level
                                                                            Except if it was Ignore its now Novel
Different        Same            Same            &         If it’s just created then Ignore it
                                           Same            &         If exists us its current interest level
Same            Same            Same            &         Must already exist from previous time
                                           Same            &         Use its current interest level
                                                                            Except if it was Ignore its now Novel

If a source stimulus is novel it will always be different. If a stimulus is “Ignore” it will always be “Same” so as to not attract attention. An efficient algorithm would not save these Ignore stimuli but create them temporarily for comparison purposes. When an expected P-Habit stimulus is not obtained then we look at the actual P-Habit stimulus obtained and find the biggest difference within it when compared with the expected one.

31st March, 2007                 P-Habits and Generalization

Upon being presented with a multi-stimuli scene we take in the entire scene as a whole. It is then that we pay attention to the parts of the scene only if they change. This strategy is consistent with paying attention to the biggest / highest level difference. This allows us to focus on the objects that are comprised of combinations of stimuli and never have to remember every primitive stimulus that makes up a scene. We then only pay attention to primitive stimuli if they change individually or we purposefully attend to them by performing a habit. Then when our attention is attracted to a combination of stimuli that all changed at the same time we can categorise it as that particular object. We would also categorise the object as a certain type of object if a subset of these stimuli match a lower level combination of stimuli associated with the type.

A possible implementation to validate this theory would be to mark all sub-stimuli (lower level stimuli) of a P-Habit as novel when the P-Habit is novel and familiar / neutral when the P-Habit is familiar.

This would mean that all the parts of an object gain the interest of the whole. An example is a particular chair and all its parts. But if a particular chair becomes interesting, chairs in general do not. That is because the interest in the type is different from the interest in the thing. Interest in a thing does not extend to interest in the type. However interest in a type (chairs) does extend into interest in all things of that type. This is because they all exhibit the same properties that distinguish the type. Such type properties are not found in the absolute values of the stimuli. These type properties are determined via the difference / change values. Type properties should not be confused with lower level absolute values that could be interesting that would extend to all objects that have this value, e.g. blue objects. In the case of chairs if you become interested in a certain part i.e. a leg, then all chairs that have the exact same leg are interesting because of their part. This is different than becoming interest in the type “chair”.

Or is it that all the properties of the type of thing are in a particular thing and when it becomes interesting the type of thing becomes interesting too. Then when you encounter another particular thing of this type you show interest but if it is not the same as the first particular thing the type becomes uninteresting.  But the first particular thing retains its interest as do many of its parts / properties that are not necessary to determine its type. This sounds more plausible.

1st April, 2007                       Primitive Stimuli feelings

If the interest levels of P-Habits which are the combination of stimuli keep changing then the lowest level stimuli, primitive stimuli must oscillate even more frequently because they are incorporated in numerous P-Habits. Thus it might seem reasonable for primitive stimuli to become P-Habit permanent just like they may also become S-Habit permanent. This would give them an unchanging interest level of neutral. Right now I can’t think of what criteria or event would cause this to happen. – See 19th April 2007

                                                Parallel A-Habits

Action sequences can be done in parallel provided they don’t overlap on the use of devices. They may overlap on the use of senses. The learning to do parallel A-Habits starts by sequentially starting several A-Sequences. Each one started must be learnt and be done at the subconscious / background level so that it can continue while concentration can begin another one.

                                                Permanent stimuli

I have tried an alternate approach to what to do when stimuli become permanent. This involves storing them in LTM as soon as they become permanent. The LTM patterns look more consistent than before although there are a few bugs to be worked out yet. Copy (7) from today was based on the old strategy with all single sense test cases working and it behaving correctly for multi-stimuli situations.

3rd April, 2007                      Expecting Permanent Stimuli

Yesterday I realized that permanent stimuli have expectations just like regular ones and when those expectations are not met I need to store the permanent stimulus plus its unexpected one. This has resulted in quite different patterns for the test cases. I have extended test case #7 & #8 to test some of the changes.

5th April 2007                       Repeating learnt actions

I’m now trying to get test case #13 working as it used to. The idea is that if in the process of exploration an action A in situation X is rewarded with a goal Y then there is a desire to repeat action A in situation X to re-experience the goal Y.   This interest propagates backwards to an interest in re-establishing situation X so action A can be done to get Y again. Now if the reward from the goal Y is a reward of novelty having done this action sequence once, Y is now familiar and has lost its novelty. What is left to motivate the repetition of the action A once situation X has been reached? Is there something about actions such that they always get repeated at least once in order that they get learnt? Is this what practice is all about? Is this what has to be done to an action sequence before it can be started and performed subconsciously?

If the feeling of novelty were stored as a stimulus in parallel with the Y then the desire to repeat the action would be based on the desire for the novelty feeling. When this is not obtained the second time action A is performed in situation X there would be no desire to do it a third time. But if each time action A was done another novel stimulus occurs the action will continue to be repeated. 

It may only be action sequences that can be performed subconsciously, not primitive responses. Action sequences require 2 or more cycles to be done and even though there is interest in performing them, that’s why they are started, they do not need to be concentrated on to “prove” they work.

                                                P-Habit Interest

I think P-Habits created from 2 source stimuli where one is different and the other same should be marked Difference = Ignore and Interest = Unknown. Then at attention paying time only differences are checked, not interest levels. The interest level then only gets set if the stimulus becomes conscious. Then I realized that a difference of ignore is a difference of zero = same. I made this change to Adaptron – only check difference for attention attraction – and it works fine. There is no involvement of interest in attracting attention. This may be different for emotional feelings. They may be involved in attracting attention. So now Interest = Ignore for a P-Habit means the interest level is unknown.

                                                A-Habit Interest

So if an A-habit is associated with the feeling of interest that was experienced when it was done the 1st time it can become neutral when the expected familiar goal is obtained the second time. If for some reason the expected goal is uninteresting when obtained the second time the A-Habits interest level is also neutral (don’t repeat). This is because a goal stimulus’s interest level can change for other reasons. When in situation X a second time with an A-Habit interest level of neutral it is possible that the expected goal has become interesting and this may trigger the action A being performed. An action interest level can’t become boring. Possible combinations are:

A-Habit Interest        Goal’s Interest          Action done?
Interested                  Interesting                 Yes
Interested                  Neutral                       Yes
Interested                  Boring                        No – fear of result
Neutral                       Interesting                 Yes
Neutral                       Neutral                       No
Neutral                       Boring                        No

10th April 2007                     A-Habit Interest

I had three more rows on this table for A-Habit Interest = Boring but decided that was not possible. However now that I have run Adaptron it appears that an A-Habit that results in a repeat of the trigger stimulus should be marked boring just in case the Goal stimulus (= the trigger stimulus) becomes interesting. I also feel that there are 3 actions that can be done, Do it, Don’t Do it (freeze) and Try something new (flee, reflexive response). The new table is:

A-Habit Interest        Goal’s Interest          Combined Interest   Action done?
Interested                  Interesting                Interesting                Yes - Do it
Interested                  Neutral                     Interesting                Yes – Do it
Interested                  Boring                      Neutral                      No – fear of result
Neutral                       Interesting               Interesting                 Yes
Neutral                       Neutral                    Neutral                       No interest
Neutral                       Boring                     Boring                        Reflexive response
Boring                        Interesting               Neutral                       No – not interested in repeat
Boring                        Neutral                     Boring                       Reflexive response
Boring                        Boring                      Boring                       Reflexive response

11th April 2007                     Attention algorithm

I need to refine the attention algorithm for habits that complete. If an unconscious habit completes the stimulus is used up and does not attract attention. If an unconscious habit uses a stimulus and continues the stimulus is used up and does not attract attention. If an unconscious habit fails there may be a stimulus left that was not used. When a conscious habit (one at concentration level 1) completes attention should be paid to the expected goal stimulus matched. I need to make sure there is only one conscious habit currently operational. The habit may have several expectations but they all started with the same trigger stimulus. When a conscious habit continues attention should be paid to the stimulus that was expected. When a conscious habit fails then attention is paid to what was obtained instead or the difference between the expected and the actual stimulus.

So a stimulus is used if it matches an expected stimulus (goal or trigger) of any habit (conscious or unconscious), or in an A-sequence. If they are all used then pay attention to what the conscious habit was expecting and just got.

An unused one can be caused by an unexpected second / goal stimulus or the recognition of a sequence (S-Habit) (familiar or novel)

1. Perform S & A-Habit matches.

2. If there are unused ones then

  • Form the group with the shortest context
    • If there are unused ones with any changes / differences then
      • Form a P-Habit stimulus, which will attract attention. Stop doing any conscious habits. We have unexpected stimuli.
    • Else if there are any that are different, match the context but used
      • Pay attention to the largest one
    • Else if not concentrating and there are unused ones with no change then
      • Pay attention to the largest one.
    • Else concentrating but have unused stimuli so
      • Pay attention to same sense / feature as expected.

3. Else there are no unused ones so

      Pay attention to the one that was expected; (goal if complete, trigger if continuing).

16th April 2007                     Attention algorithm

I’ve got the else part 3 working and now for the part 2 where there are unused stimuli. Similar to P-Habits we want the largest sequence to attract attention. The one with the shortest context best represents this. Also we want only one that is unused. We also only want one that is different / changed because the senses have used up the ones that have not changed.

17th April 2007                     Attention Algorithm

I have been refining the attention algorithm and it now works as follows:

1. Perform P-Habit matches. As this is happening stimuli are marked as used if they are primitive or P-stimuli that are different and combined to form a P-stimulus that is different.

2. Perform S and A-Habit matches. If any stimulus is used in any way (matches expected) then it is marked as used.

3. If there are stimuli that are different / changed as detected by senses or P-Stimuli formation that are also unused ones then (note, all S-habit stimuli are different) they potentially attract attention so:

  • From the group of different, unused ones with the shortest context (because these are the longest recognized sequences)
    • If there are non-permanent ones then
      • Pay attention to the last one
    • Else if there are any that are permanent then
      • Pay attention to the last one
  • Else if there are still unused ones with a difference they must have a longer context – this only occurs far out in test cases #3 and #4 so then
    • Pay attention to the largest unused one.

4. Else there are no unused different ones i.e. all the different ones have been used or there were no different ones so

  • If not expecting any particular stimulus because not doing any habits then
    • Pay attention to the biggest one
  • Else if any habit got its goal then
    • Pay attention to the first habit that succeeded
  • Else
    • Pay attention to expected stimulus of 1st habit or stimulus on same sense / feature as expected by 1st habit.

This more closely matches the thinking from 11th April and goes back to the modified strategy of 21st March where it’s the properties of the resulting stimuli that attract attention not the properties of the habits being performed.

18th April, 2007                    Thinking

I’ve been working on thinking today and realized the following happens when thinking about consequences of actions. If the stimulus just perceived is boring you will not necessarily perform a reflexive response if you recognize possible immediate gratification from an action-A or you think ahead and see that a subsequent possible action-B may end up with something interesting. Also you may go ahead and perform action-A even though its goal may be boring as long as a subsequent action-B is possible that results in an interesting subsequent goal. This needs thinking. However if the stimulus just perceived could result in immediate gratification due to some action-A then thinking ahead and finding a possible subsequent action-B may end up with something boring and does not stop you doing the action-A.

                                                Pleasant / Unpleasant

The same logic applies for emotional feelings, just need to replace the words as follows.

Novelty / Familiarity feeling         Emotional feelings              General term for both
Interesting, novel                         pleasant, good                    rewarding
Uninteresting, familiar                 unpleasant, bad                   punishing
Expected boredom                      Fear                                     unwanted, avoid (action)
Interested                                    Desire                                  wanted, pursue (action)
Surprise                                      Pleasure                               reward
Boredom                                     Pain                                      punishment

19th April, 2007                    Generalization / Specialization-Discrimination

The table from 10th April, 2007 should be used to address the problem documented on Dec 4th, 2005.

From the 31st March and 1st April 2007 notes I now propagate the feeling assigned to a P-Stimulus to all of its parts, all the way down the tree to the primitive stimuli. An alternative strategy is to only propagate the feelings down one level in the tree. The lower levels then (hopefully) become neutral and don’t keep changing / flip-flopping back and forwards. This is rather like they become permanent.

                                                Interest and expectation of Interest

A clicker does not provide anything rewarding. In itself it is unrewarding. But it can be associated with reward in that it signals that the action done was successful / correct / matched desired – expected goal / outcome. Thus the clicker appears to be associated with the idea / expectation of reward, not the reward itself. So if we get into the ability to store feelings just like any other stimuli then maybe the expectation of reward should be storable and recallable. This would also seem to imply that expectations of stimuli in general should be storable and recallable. But this idea has not yet appeared necessary because each time an idea (an expectation) of a stimulus is generated it is matched with the stored stimuli to find subsequent expected stimuli. It is not matched with stored expectations of stimuli. This is because thinking uses real experiences as the basis for modelling the world. Reflecting this back on the expectation of a feeling it must be that the clicker is associated with a rewarded stimulus in experience and every time the clicker is heard the expected reward is recalled. And the clicker does not lose its reward association provided all subsequent uses are associated with stimuli with neutral feelings.

                                                A-Habit feelings

This would mean that for an A-habit if the goal were not itself rewarding but instead associated with reward (has an expectation of reward) then the A-Habit is worth repeating and gets a wanted feeling assigned to it. This is the feeling of the A-habit described on the 10th April which takes on the three forms of:

1/ wanted / to be done/ desired / repeat when triggered but only if expected outcome is expected to be neutral or rewarding

2/ neutral / can do / know how to / do when triggered if expected outcome is expected to be rewarding / don’t do if expected outcome is expected to be neutral or unrewarding.

3/ unrewarding / don’t do even if expected outcome is expected to be rewarding. Thinking about a rewarded outcome does not appear to overcome the 3rd A-habit feeling like it does overcome the situation described yesterday where the outcome is unwanted but expected subsequent actions are expecting a reward. A-habit feeling #1 is what the clicker imparts to the A-Habit. Similarly a “No” signal should impart feeling #3 to the A-Habit.

20th April, 2007                    Thoughts

Do you remember thoughts? If so are they retrieved from storage or are they recreated based on remembering the experience that triggered them. We certainly say, “I remember thinking…”

21st April, 2007                    A-sequences

My A-Sequences are currently working in subconscious mode but they breakdown / are interrupted by conscious actions if they get too long. Thinking turned on extends them one more response but they still breakdown. This is because the only thing holding them together is the desire for novelty / interest and every time they are done the stimuli lose any novelty due to repetition. This should not happen with emotional feelings. They will be held together by each A-Habit being marked worth repeating because they expect to get a goal that is associated with some pleasant stimulus as documented on 19th April. By using the “Yes” signal on one’s own actions one can practice a long A-Sequence until it is perfected.

                                                No Stimulus - Habituation

When we close our eyes we have no visual stimulus. When there is no noise there is no auditory stimulus. It’s not a stimulus with a particular value of zero; it just is not being produced. But isn’t this the same as a value that has been habituated by the senses? There is no difference signal being produced. At the start of an Adaptron run the first stimulus should act in this fashion and have no difference indicator.

                                                Changes

All the changes for an object occur simultaneously, as previously noted, but also the amounts of these changes are the same for an object.

Neotron might be an alternative name for Adaptron

23rd April, 2007                    Senses

Following on from May11th 2005 notes: A graduated stimulus is not usually linear it varies over multiple orders of magnitude and the resolution at any point follows - Fechner’s Law.

The five external environment senses are Hearing, Taste, Smell, Touch and Sight. The Internal body senses are Kinesthetic, Balance, Hunger, Thirst, and Time.

Hearing:         Pitch like the keys on a piano go from 27.5Hertz to 4186Hz giving a range of values and each pitch has a volume that could be measured in decibels which is also non-linear. Normal piano volume is in the 60-70db range. Two sources – ears.

Taste:             probably 5 chemical reactions with a large range of intensity values

Smell:             possibly 1000+ chemical reactions with a large range of intensity values

Touch:            a 2 dimensional array of locations each with a range of pressure values and a range of heat values.

Sight:             A 2 dimensional array of locations each with a range of brightness values and a range of colour values. Two sources – eyes.

Kinesthetic:    Multiple joints / muscles with a range of tension values.

Balance:         Maybe multiple directions with a range of tilt values?

Hunger and Thirst: Each a single stimulus with a range of values?

Time:              challenging.

I am going to remove the concept of feature and instead have multiple senses each of which has a one-dimensional array of sensors. Each sensor can detect a range of values. Then for a given sense it must be specified whether there is any order to the one dimension (is it discrete or graduated?) and then whether the sensors produce a discrete set of values or a graduated range of values. All sensors in a sense must be equally discrete or graduated.

So if on the sense of hearing we want to provide Adaptron with pre-recognized stimulus values such as Meow, Bark, Slam, Splash we could have just one sensor in the one-dimensional array which has 4 possible discrete values.

If we want to be able to detect a Meow and Slam at the same time the 4 values could represent 4 discrete indices (sensors) in the one-dimensional array and each sensor’s value could be a discrete 0 or 1 to indicate it has been detected.

If we had a volume reading for these 4 sound types we could indicate that the 4 discrete sensors are graduated with a range of volume values. An example on a robot would be 20 motors controlling different parts. Each motor would have its own discrete sensor that measures the graduated rotational angle.

If we are listening to a piano and are not interested in the volume, just whether a key has been hit we would use a one-dimensional array of sensors graduated according to piano key frequencies and use a discrete 0 or 1 sensor value to indicate the key. This might be used on a robot that has a belt of many pressure sensors around its girth, which it uses to detect if it has bumped into a wall. If the sensors had the sense of touch they could provide discrete readings of the material they are in contact with, e.g. metal, wood, cloth, plastic, glass, sponge etc.

If we want to represent animal hearing capability we would use a graduated one-dimensional array of frequencies with sensors that are graduated with a range of volume values.

24th April 2007                     Interest = Change

Exploring the use of the change / difference as a source of interest levels. The idea that a stimulus that is different from its previous one has a certain difference / change and this attracts attention if not used by a habit. And its difference has to be greater than the concentration level to distract the current habit. So if we are doing an A-Habit with concentration of 3 we are interested in getting its expected goal and will not let any stimulus distract us with a change of 2 or less, nor any stimulus that is used by a habit. To determine the concentration level of an A-Habit, which is the expected interest, we add the interest in the goal stimulus and the interest in the A-Habit itself. The interest in the A-Habit itself is the experience interest minus the expected interest.

The reason we perform an action is to get the goal stimulus and obtain a certain amount of change. It fails if we don’t get the goal and remains as an A-Habit to be done. But if we do get the goal but not the amount of change we adjust the A-Habit interest up or down. If we get the goal and the amount of change it becomes an uninteresting A-Habit that will only be repeated if its goal’s interest improves. We also perform an action as a reflexive response. Here we expect at least a minimum change but no expected goal stimulus.

25th April 2007                     Experience Interest

The new A-Habit interest, which is assigned to the trigger stimulus, is the experienced interest minus the expected interest. The expected interest was the A-Habit’s interest plus the expected goal’s interest. The experienced interest is the actual goal’s interest plus the change experienced on this stimulus.

26th April 2007                     Motion & Object detection

If we have a graduated one dimensional array of sensors that take on two discrete values of 0 or 1 we should be able to devise an algorithm that finds objects from combinations of sensors that are on. I programmed a motion simulator to see how the eye detects line motion in a single dimension. First any sensor whose value does not change is ignored and is considered as ‘background’, not part of an object. Then for all those sensors that have changed from 0 to 1 find the closest that has changed from 1 to 0. The nearest (closest / moved the least distance in the one-dimension) becomes an object that has ‘moved’. However if two or more have moved the same amount in the same direction even though the distance is not the least then they are seen as belonging to the same object. However the further they are apart the less accurate is the resolution of the actual distance they are apart due to Weber’s law. For example Dots 1, 2 and 3 have distance changes to 3 other locations as:

                     D1       D2       D3
Distances      1          6          7
 to other        6          11        12
 points          10        15        16

D1 and D2 are seen as a pair, which has moved a distance of 6 and kept their separation of 5.

28th May 2007                      Graduated array of sensors

I have found that given any snapshot of values from the array of sensors it is first necessary to recognize the objects. An object being recognized as a grouping of adjacent sensors that all read the same value (a 0 or a 1). The algorithm that does this and represents the objects as binons makes multiple passes over the array. Whenever a change in value occurs between sensor #1 and sensor #2 an object is recognized based on the second’s value. But if sensor #2 is followed by another sensor #3 with the same value, the pair of values is recognized as an object whose value is that of the sensors (see note below). Habituation causes us to skip sensor #3 if another one of the same value (sensor #4) follows it. If however it is followed by sensor #4 with a different value a change is detected and we start the loop again for this new object. Successive passes reduce pairs of values to a single object thus achieving the recognition of the following sequences as the given objects.

Length of sequence of same values / objects

1                                         single sensor object
2                                         pair object
3 and 4                               small object
5, 6, 7 and 8                       medium object
9, 10 … 15 and 16             large object
Etc.                                     just like Weber’s law – resolution is not as good  for larger sizes

Note: The current algorithm uses the object ID as the value to detect repetition of values on 2nd and subsequent passes. This works because an object consists of a binary hierarchy of objects that are finally grounded on the values. And change amounts are not recorded for the objects – so interest cannot be determined. Objects are created as sensor independent but for each snapshot the objects occur at particular sensor positions. A single object may occur several times in a given snapshot. It is at the sensor position that the change / interest in the object occurs.

But now the different combination of objects that make up the snapshot must be recognized. If we had two sequential snapshots we could pick out the combinations of objects that move together to recognize sub-objects. With just one snapshot do we generate all possible combinations including the complete scene or do we just generate the whole. One would expect the former because if a sub-object pattern occurs twice or more in the snapshot it should be recognized as the same object. Also if it repeats adjacent to itself the repetition habituation from above should kick in.

30th May 2007                      Repeating patterns

The following input sequences should be recognized as the following objects. Inputs are 0 or 1 and objects are 2, 3, 4 etc.

000111           0001000        00010111      000101000    0001001000    00011000
2  23  3          2  212  2        2  2 3 4  4      2  2 3 12  2    2  212  12  2    2  23  2  2

Pairs of the same value are recognized as an object and a change causes the creation of an object recognizing the pair. The algorithm for this is:

  • If 1st = 2nd then
    • Create the pair object,
    • Remember the pair object,
    • Note the source value,
    • Adjust pointer over 2 so repeat above at 3rd.
  • Else 1st <> 2nd then
    • If last stored a pair object and the 1st is the same as the source then
      • Save the pair object for the 1st,
      • Adjust pointer over 1 so 2nd becomes 1st in following steps.
    • If 1st = 2nd value then
      • Repeat above at 1st.
    • Else 1st <> 2nd so if 2nd = 3rd then have a pair
      • Save the 1st value
      • Adjust pointer over 1 and repeat above at the 2nd.
    • Else 2nd <> 3rd then
      • Create an object from the 1st and 2nd,
      • Adjust pointer over 2 and repeat the above at the 3rd.

[The first test run of software that eventually became Perceptra 1D ]

3rd June 2007                      Graduated sensor values with a graduated sense

I initially thought that the pattern 000101000 should produce a different object from 000202000 and 000404000. The value of the sensor reading (intensity, volume, brightness or whatever) would result in these being observed as three different objects. But on further thinking I realized they should be recognized as the same object with different values. But that leads to the question is 000104000 the same as 000202000 where the parts have changed their value? The answer would appear that it is two objects because their values change independently. So are 000102000 and 000203000 different objects? Here the change amount is the same so it is the same object. This just repeats my earlier statement that all the parts of an object change together at the same time by the same amount.

6th June 2007                      Object recognition

I’ve got a pretty good object recognition algorithm working. It recognizes 000202000 as the same as 000404000 because when one of any two adjacent objects has a value of 0 it uses a value of 1 for the non-zero object. It progresses from left to right and prefers to recognize X0 adjacent objects before 0X ones. It also assumes each sensor is the same object but with a value being its measurement. Each new binon represents an object formed from two adjacent source objects. The bucketed value of the difference between the values of the two source objects goes into the unique identification of the new binon. So a new binon is the combination of the two source objects and the bucketed difference between the two source object values. Each such object is then found at a particular position in the “field of view” and has a particular value at that position. Thus 005200 might produce:

Object # 16 17 16
Position:  1   3   5
Value:      0   5   0

Where 16 is a pair of object #1s with value 0. 17 is a pair of object #1s with value 5. Object #17 is based on a difference of –3 while object #16 is based on a difference of 0.

Thus objects / binons are position independent. They are based on structure and relative (intensity) values.

                                                Motion recognition

The task of taking two sequential “frames” and comparing them to detect the objects is now possible. The first step is to disregard all the objects that have stayed in the same position with the same value. Only changes attract attention. But then there are multiple things that can happen in many different combinations. An object can move. A new object may appear. An existing object may disappear. The (brightness) value of an object can change and it can also move or the value can change but not move. And all this can happen to as many objects that are present. One can move in one direction while another moves in the opposite direction. What does the human brain do in these circumstances? In the real world most of the time only one object moves or changes its value, or the entire scene gets brighter or moves due to a body motion. One approach might be to pay attention to the biggest change, where biggest is hard to define unambiguously right now. It might be the largest combination of objects that move together or change their values the same amount. It might be the largest move and not the size of the object. It might be the largest value change and not the object size. This would mean a newly appearing object might get the attention rather than existing ones that moved. Then there is the concept of wider or narrower as though the object is coming closer or receding.

7th June 2007                      Object recognition

An object is recognized as a grouping of part objects in which all the parts change together. See 18th Dec. 2006. They move the same amount, their intensity changes in unison. Their relative intensity is always the same even if it appears two or more times in the one scene. The distance the parts are from each other may change but the relative distances they are apart is always the same. The parts may have a size and this may change, but their relative sizes to each other must not change if they are to be parts of the same object. So a part in a graduated sense with a graduated array of sensors has intensity based on the value of the sensor, a position based on which sensor and a width based on how many sensors are used to detect the object.

The most primitive objects start when there is a change in intensity / value and continue across multiple sensors reading the same value until there is another change in value. This next change in value starts the next object.

It is possible that the 000101000 and 000404000 are better recognized as the same object made up of two primitive objects separated by a distance of 2 rather than an object made up of ‘10’ and then combining this with the 2nd 1. This would also allow for objects to be transparent with objects moving inside the boundaries or in front of a larger object.

8th June 2007                      Object Identification

Currently an object is identified as comprised of two source objects that have a difference in intensity that has been bucketed. It is now necessary to include in that identification something about the distance they are apart and their width. For the intensity difference I use the two source intensities / values and bucket the result. Then the intensity / value of the object is set to the intensity / value of the 1st source object. For distance I could use the distance between the two source objects and bucket it. Then the position of the object could be set to the position of the 1st source object as it is currently. When an object moves to a new position / sensor the relative distance apart of the two source objects stays the same. For width I could use the bucketed difference between the widths of the two source objects and assign the width of the 1st source object to the width of the object. When an object moves towards or away from an observer the width of the parts increase or decrease together but so too does the distance the source objects are apart. And when an object flies apart or comes together the width of the parts may not change but the distance between the parts does but maintains the same relative distances from each other. And then there is an object that expands or contracts in which the distance apart of the objects does not change but each part expands or contracts in unison.

Because we want to capture the relative intensity of the two source objects we should not be subtracting the two intensities and then bucketing the result. We should be bucketing the two intensities and subtracting one from the other. This is because A/B is Log (A) - Log (B) and not Log (A-B).

But I am still faced with the problem of the fact that there are two ways to identify an object made up of 3 parts. Either (1) 1st and 2nd parts combine to form an object which is then combined with the 3rd part or (2) the 2nd and 3rd parts combine to form an object which is then combined with the 1st part. How can I recognize these two combinations as the same object?  If I use a left to right preference and only use the (1) combination of three parts but also generate the 2nd and 3rd combination as another object then I have covered all the bases. I would have all pairs of objects identified plus the combination of 3. This approach is similar to Jan 9th 2004 in which lowest level parts are combined at many higher levels. This will work for combining objects that are adjacent to each other but not when there is a gap (another object) required between the 1st and 2nd or 2nd and 3rd parts.

How do we handle this as humans? We concentrate on a small range / area of the scene at any one time and a change in the scene attracts our attention at which point we concentrate on this new area. Within that area of concentration we perceive more detail than at the periphery. For sight we move our eyes to the area but for hearing and the sense of touch we mentally move our concentration area.

9th June 2007                      Background intensity

One thing that will help with object recognition is the use of background intensity. This is the most frequently occurring value. Most of the time it would be no signal from the sensors. But if we are to recognize no signals in a sea of loud signals the loud signal value must be seen as the background. Similar to the ambient noise level. Then sensors with values that are the same as the background do not contribute towards identifying an object. This should reduce the number of objects and allow for “seeing” black objects on a white background or the same object in white on a black background.

Now a group of objects stands out as objects that are adjacent without any background values between them. And larger disconnected groups exist where the width of the background values between them is small relative to the width of the background values on either side.

19th June 2007                    Attraction priority

The order in which changes attract attention is most likely:

  1. Position change
  2. Separation change
  3. Intensity change
  4. Width change

A separation change is perceived as a change in position for some of the parts. This would mean the part that moves the furthest would attract attention as a separate object. But we want to perceive the same object if the separation changes in unison with the width because it is coming toward us or going away.

Objects should be identified as having two parts with:

  • a particular ratio of (or relative) intensities,
  • a particular ratio of (or relative) widths, and
  • a particular ratio of (or relative) width to separation ratios.

If width stays the same but separation increases then we start seeing the parts that make up the whole as separate objects. If the separation stays the same but the width changes then we see an object whose parts all expand in unison, or would we? Is 00110000001100 the same object as 0011110000111100 or is it the same as 001111110011111100?  No I think they should be different objects. This would mean that an object should be identified as having two parts with the 3 ratios specified above.

Then when an object X is recognized it has an intensity value, a position and a width. Its intensity value could be kept as the Log (base 2 = bucketing) of its actual absolute value. Its width could be kept as its actual absolute width, which is Object#2 position – Object#1 position + Object#2 width. Its position would be kept as its actual absolute position. The relative intensity of object X (which helps to recognize it) is Log(Object#1 intensity) – Log(Object#2 intensity). The relative width of object X (which helps to recognize it) is Log(Object#1 width) – Log(Object#2 width). The relative width to separation ratio of object X (which helps to recognize it) is Log(Object#2 width) – Log(Object#2 position - Object#1 position).

If two objects are adjacent to each other and the intensities are 22 and 25 we can notice the difference because they are simultaneously adjacent. When they become brighter they are 102 and 105 and we can notice the same simultaneous difference. But the change from 22 to 102 is perceived as a sequential change in magnitude from 5 to 7 (bucket 5 = 17->32, bucket 7 = 65->128). This seems to imply that simultaneous direct sensor readings can be compared and the difference determined at any absolute value. The difference between the two intensities though would be bucketed (=3) for later object recognition, as would the sequential change in intensity (=2). It would seem that as soon as the information needs to be recorded / kept / represented it is bucketed.

This would mean that the sensory objects in a frame and the immediately next frame keep their absolute values for doing comparisons but all differences obtained are bucketed. The objects are identified by these bucketed differences. For detecting moves, intensity changes and width changes absolute values are used. So now an object’s relative intensity should be Log(Object#1 intensity – Object#2 intensity). An object’s relative width should be Log(Object#1 width – Object#2 width). However an object’s width to separation ratio should be Log(Object#2 width) – Log(Object#2 position - Object#1 position) as before. An object’s intensity, width and position can be those measured by the sensors for object #1.

I currently produce an object to represent the entire frame needed for the 1st one that is experienced. This is done by combining every second object successively in layers until I end up with one object. Rather than using every second object I could use some algorithm that groups objects based on relative separation distances.

Experimentation has shown I need to use relative width of Object#1 to Object#2 rather than the difference in width.

After an object has been recognized as a result of all of its parts moving the same amount and it has been given an object ID, what gets presented to consciousness? The object ID with Sense and Sensor position. This will allow future habits to pay attention to the sense and sensor expecting the object. But also consciousness is presented with the change information. Changes can occur in one or all of the following; amount of the move, the change in the intensity and the change in width to separation ratio. Whichever change is the biggest between two frames will cause the object to be attended to.

21st June 2007                    Motion detection

I developed a motion detection algorithm for change in position / sensor for objects but it failed on a very simple test. I’ve read about Reichardt elementary motion detectors and Adelson and Bergen’s more elaborate detection techniques in vision. Some observations are that motion is detected by finding the closest second frame object (least change in position) with the least change in intensity and least change in width. But it does this for the largest grouping of objects first. So a new algorithm could start with the biggest objects in frame 2 and see if they were in frame 1 and find the least position / intensity / width changed one. This would work its way down to whatever remaining smaller objects it finds.

However this algorithm needs all possible objects to be present in both frames. The current object-grouping algorithm based on proximity does not produce all possible groupings. An algorithm that combines adjacent objects would suffer from exponential explosion but produce fewer objects if no holes were allowed. That is, it would combine objects 1 & 2 and 2 &3 but not 1 & 3. If N objects are involved it would produce N * (N-1) / 2 combinations.

Next the question arises as to how to calculate the change in position, width and intensity? Do I use absolute changes in position such that a change of 34 for object #1 and 35 for object #2 are seen as different changes or do I deal with bucketed values of the changes in which 34 and 35 are both distances of order of magnitude 6? This may be taken care of based on the width to separation ratio of the object that is the combination of object #1 and object #2. If the two are thin and separated widely enough they will both look like they moved the same distance because the combined object is seen as moving, not two individual lines.

Motion detection also has to be based on the fact that it is of very low probability that new objects appear or existing objects disappear. Or put another way conservation of objects is most likely.

30th June 2007

See the 2007 Scientific Research and Experimental Development (SR&ED) tax credit claim.

Details about the Canadian Revenue Agency’s SR&ED tax incentive program are here.

4th July 2007                                    Attention to Sensors

A recognized object has values for:

  1. its position which is the sensor to focus on (pay attention to) to find it.
  2. its width / size which is the range of sensors needed to find it.
  3. its intensity / volume which is what level to focus on.

I have traditionally been thinking of attention as the point of concentration to obtain a stimulus. It is reflexively attracted by an unexpected change and it is purposefully focuses when performing a habit. I have previously only thought of it as consisting of a sense and a sensor within that sense. The sensor would be a frequency sensor for hearing, a left/right & top/bottom position/direction sensor (retinal) for vision, a location sensor for touch etc. But the sensors represent only one dimension of the sense. The 2nd dimension is provided by the reading / measurement / value / intensity of that sensor. When objects cause the sensors to detect a change in their readings our attention is attracted not only to a single sense and sensor but also to a range of sensors that have been triggered and to an intensity level that has occurred. Thus when we focus our attention it is:

  1. on a particular sense or combination of senses (as in P-Habits),
  2. on a particular sensor
  3. over a range of sensors and
  4. at a particular intensity level.

Then at this particular focus point / focus area an object is recognized. The object has an identity, is composed of two sub-objects, with a relative intensity, a relative width and width to separation ratio. For final sub-objects the relative intensity and relative width are zero and the width to separation ratio is 1.

Should the attention be focused on a particular intensity or more likely an intensity range based on the expected objects relative intensity at a particular intensity level? This would effectively say we attend to a box of particular size and at a given position in a two dimensional space with axes of sensors and intensities.

6th July 2007                                    Discrete senses

On 23rd April 2007 I described various combinations of discrete and graduated senses with discrete or graduated sensors. If I have 4 range finders, each facing a different direction, forward, right, backwards and left they can be represented by one sense with 4 sensors that are discrete but their readings are graduated. Because the sensors are discrete they are independent and all possible combinations of them must be able to be attended to. There are 11 possible combinations without gaps. I’m trying to devise an algorithm that does not need to create all possible combinations as binons to represent what is perceived. The combinations can be identified using a 4 bit binary number where a 1 bit indicates that the associated sensor is relevant to the combination. This would mean that the S-List would contain for each sense the one combination that has attracted this sense’s attention, but a complete list of all sensor values in case an active habit wished to focus on a different combination.

Based on the 4 possible combinations of discrete / graduated used for senses / sensors the following schemes could be used.

  Sensors – Intensity discrete Sensors – Intensity graduated
Individual  Sensors – discrete sense

Object = one for each value on each sensor. Each symbol is an object.

Attention to combination of sensors and the symbolic / intensity values of each which have changed at the same time.

Change = 0 or 1 = same or different.

Object = only one lowest level, the same on all sensors

Attention to combination of sensors that have maximum change and the intensity of each.

Change = max intensity change of combination.
Dependent Sensors – graduated sense

Object = 2 parts + two symbolic / discrete intensity values, relative width and relative width to sensor separation.

Lowest level, no parts, the two symbolic / intensity values are the same symbolic value & the relative width = 0, width to sensor separation = 1.

Attention to sensor & range and ?

Change = Sensor position, width + 0 or 1 = same or different symbolic / intensity values.

Object = 2 parts + relative intensity, relative width and relative width to sensor separation.

Lowest level, no parts, relative intensity & width = 0, width to sensor separation = 1.

Attention to sensor & range and intensity & range.

Change = Sensor position, width and Intensity

A possible implementation for independent discrete sensors would be for each sense to have a string of 0s and 1s (length = number of sensors) to represent the combination of sensors that have attracted attention. Have an equivalent string of values / intensities that were measured by the sensors and a third string of intensity changes for the sensors. The attention string would have 1’s for the sensors that combined together to represent the greatest change.

16th July 2007                      Trigger stimulus Interest Level

I thought that the trigger stimulus interest level for an A-Habit should not just be assigned equal to the difference between the experienced and expected interest levels but be asymptotically changed towards the difference.

17th July 2007                      Interest in Objects / Stimuli

Is it possible that stimuli / objects have no intrinsic interest level? It is the interest they create by being expected or unexpected and whether they are a change from the previous one experienced that drives our actions. If a non-zero change (negative or positive) were to generate an interest equal to the absolute value of that change and a zero change were to generate a boring (-1) level of interest could this be used to drive learning. The interest generated would be stored / associated with the experience and not given to the trigger or goal stimulus. Now when the trigger stimulus reoccurs the interest level in repeating the habit would be based on the amount of change experienced previously. This would be the expected interest. Then if the same goal occurs the same change occurs and there is no difference between the expected and experienced interest. Using the idea of no change / difference is boring then the new experience would be boring.

24th July 2007                      Concentration

I have a problem with concentration level, when to set it and when to neutralize it. One possible strategy would be to have many habits executing in parallel all with their own interest / concentration level. The stimulus that attracts attention is the conscious one and if it happens to be the goal of an active habit then we acquire the concentration level of that habit. With new habits started the concentration level is that of the most interesting.

But should the concentration level be used to filter out stimuli that are of low interest / change or should it be the execution of background habits that do this? Or is there a combination of both mechanisms at work? I suspect the combination.

25th July 2007                      Change / Difference

The experienced interest is usually the amount of change / difference experienced for the attended to stimulus as measured by the sensors. However if this stimulus matches what was expected by habits then the experienced interest is reduced to neutral because it did not produce a change sequentially as far as S or A-habits were concerned.

And if no habits got their expected second stimulus then the expected interest should be reset to neutral.

26th July 2007                      Change / Difference

In the pursuit of novelty Adaptron is interested in change. Thus at the sensor level a change is important while no change is not noticed. This is called habituation. When multiple sensors are involved it is those that change that detect the object. Because such stimuli are in parallel there is no chance to react between them and they immediately form permanent binon networks that represent and are used to detect the objects when they reoccur. Thus the senses can highlight what is different from one frame to the next. That is, the same object might be detected but with a different brightness / intensity, width or position. Or a different object is detected, the previous one having disappeared. The amount of change determines the interest level.

When sequence is involved the LTM is used to capture the experiences of pairs of stimuli as sequential binons (S-Habits). When the 1st stimulus of an S-Habit occurs, multiple habits are started expecting the possible 2nd stimuli with an expected interest level from experience. This is done so a change can be detected i.e. a sequence that has not occurred before. If a sequence (S-Habit) gets its expected 2nd stimulus the stimulus loses its interest because no change was experienced. That is, the sense said it experienced a change but LTM said the change was expected. If any stimulus is left over that was not expected by some S-Habit then a different sequence has occurred and that is interesting and worth recording in LTM. So we pay attention to any stimulus that still has some interest left.

Multiple habits started each have their own expected 2nd stimulus with their own expected interest. Multiple stimuli occur each with its own sensor experienced change / difference which determine their initial interest level for attracting attention. As the stimuli match the expected 2nd stimuli of the habits, the stimuli’s interest level gets reduced by the habit’s expected interest - they were expected.

But we are only conscious of the most interesting habit / expectation. It’s the one we are doing consciously. It is the only one that can be learnt. It is the one that was interesting in the past. When the stimuli arrive we are concentrating on one sense / sensors and with an above neutral concentration level. Any stimulus with interest remaining above the concentration level after matching all S-Habits attracts attention away from the sense / sensors expected. Else, the stimulus matching the sense / sensors expected is attended to. The attended to stimulus’s interest level is then assigned to this sequence. Then the LTM past experienced interest levels for this stimulus as a first stimulus are used to determine whether to perform a reflex action or repeat a past action.

The other alternative is that we are not concentrating at an above neutral level – have no interesting expectations. But we may have some neutral S-Habits with expectations. Any stimulus with interest remaining above the concentration level (neutral) after matching all S-Habits attracts attention. Else, the stimulus matching the sense / sensors expected completes the expectation and is attended to. Then the LTM past experienced interest levels for this stimulus as a first stimulus are used to determine whether to perform a reflex action or repeat a past action. This is the same as the previous paragraph with a different concentration level.

This approach modifies the attention algorithm from the 17th April, 2007 so that the used flags are not used. Instead interest levels are used.

What about a stimulus that matches an expected trigger stimulus and the habit continues? Does this habit affect the interest level of this stimulus? I don’t believe so. The interest level of the stimulus is going to be used to mark the LTM sequence’s interest level if the stimulus gets attended to and stored in LTM. If it matches a trigger stimulus this trigger could become part of the context and then stored. So the context needs to be given an interest level.

An idea I could try is to start an A-Habit whenever a reflexive response is being done with interest level = neutral = 0. It would be expecting no change, i.e. a repeat of the trigger stimulus. This may save having to check if we are doing a reflex. This could be extended to what I tried before which was to have a repeater S-Habit with neutral interest when no expectations are known in LTM.

29th July 2007                      Expected Stimulus

The execution of a habit looks like this on the habit stack.

24  -25  -14  -5                       expecting at 5
24  -25  -14   5  -6                  expecting at 14
24  -25   14  -15                     expecting at 25

So the next expected stimulus is (while scanning backwards through the list) the first negative value before encountering an adjacent negative value or the last negative value before a positive one.

If an Action Sequence is being performed the habit stack has the final goal habit on it followed by the intermediate goal and might look like

31  -32  -13  17a -18 -7         expecting at 7
31  -32  -13  17a -18  7 -8     expecting at 13
31  -32   13  -14                     expecting at 32

This works the same way as above.

After sequential habit matching has been done if there are no stimuli that attract attention then we pay attention to the expected sense / sensor. If the expected stimulus is a non-sequential habit then it finds the actual stimulus experienced and uses its experienced interest. This may equal what was expected or be a different one. Whichever, it still becomes the conscious one. However if the expected stimulus is an S-Habit then if this S-Habit can’t be found on the stimulus list we have an unexpected sequence and should pay attention to the actual stimulus experienced. Mind you this actual stimulus should probably attract attention because matching expectations of active habits did not reduce its experienced interest. For this purpose it would be necessary to also keep track of the final expected stimulus of each habit.

30th July 2007                      Sequential Interest

The sensors determine the difference of a stimulus, which then determines its interest. But the interest is reduced by expectations of S-habits being matched. This difference works fine for the lowest level primitive stimuli. But when S-Habit stimuli occur there is a difference with the last LTM conscious stimulus to consider. The LTM is effectively the sensor for S-Habits. So stimuli can have a sequential interest level. This becomes important when a stimulus is permanent because we want to detect unexpected sequences. This would be done by setting a sequence interest level based on a difference from the last attended to stimulus and reducing this by habit matching of expected stimuli. Then the unexpected stimulus would have a remaining sequential interest level.

31st July 2007                      Experienced versus Expected interest

When a sequence first occurs, after the trigger you expect nothing and experience Interest (due to a novel stimulus). The sequence then gains an interest in repeating = Experienced – Expected. The second time you repeat it because there is interest in doing so and you should once again get the experienced interest due to a change in stimulus. The interest in repeating = Experienced – Expected and will be neutral. But the expected interest for this sequence is still that the second stimulus will cause interest due to it being different from the previous. Thus each sequence has an interest in being repeated and an expected interest if it is repeated. Is this expected interest an interest of the second stimulus or a property of the sequence? It is a property of the sequence because it can only be experienced given the first trigger stimulus.

6th Aug, 2007                       Sequential Interest              7:00pm to 10:00pm

I have also been using Sequential Interest as the experienced interest of an S-Habit when it is recognized. This is done by; comparing the S-Habit with the last LTM stimulus attended to. If it has context stimuli it probably should be compared with the last context stimulus instead. Permanent stimuli that contribute to the formation of a new S-Habit can also have experienced interest and an interest in being redone. Thus it is possible to simultaneously have a novel S-Habit with sequential interest and its second source stimulus with experienced interest and thus both with the same redo interest. I need to either reduce the source stimulus’s experienced interest because it was used in matching a habit or increase the new S-habit’s interest so that attention picks the S-habit.

I am currently using the idea that attention is attracted to any stimulus whose experienced interest is higher than concentration level. But I think I should be using any stimulus whose redo interest (experienced-expected) is over the concentration level because concentration level is based on interest of redoing, not on experienced interest. This might solve the above problem because the new S-habit will have a redo interest = its experienced interest = its sequential interest while the source stimulus should have a redo interest = its experienced interest – expected (same as experienced) which will equal neutral.

12th Sept, 2007                    Storing in LTM

The criteria for storing stimuli in LTM would seem to be if there has been anything learnt then store it. To do this Adaptron currently stores stimuli in LTM because they have been attended to. And they have been attended to because of the change or sequential interest i.e. unexpected. The only reason it does not immediately store a stimulus in LTM is if it is permanent. It then holds on to it just in case it is followed by another permanent stimulus and the two form a known sequence = an S-habit. My test run of 17th Aug 2007 tries to deal with the S-Habit AA that occurs when A becomes permanent. The question is when to store the first experienced AA sequence in LTM? In this test run after the reflexive response “b” there is no expectation and if A occurs there is no difference interest and no sequential interest but A will become permanent.

1/ One could use the rule as I have done in the past and said as soon as a stimulus becomes permanent, store it in LTM. However this does not work if AB exists earlier in LTM and B has become permanent and after the “b” response we get AB. The AB will not collapse but it could still be marked as already experienced / ignore and use a later AB experience for collapse into an S-Habit.

2/ A better rule would be based on expectations. If the stimulus that attracts attention was not expected (i.e. does not match any expected next stimuli) then it needs to be put in LTM because this sequence has to be learnt. This would imply that after any reflexive response we always store the next stimulus that attracts attention because there is no expected stimulus. This also means that after a reflexive response no S-Habit can be recognized because the first stimulus is always stored in LTM. If a response is to be learnt to result in an S-Habit then this can only be done when the A-Habit is being repeated / practiced.

17th Sept 2007                     Interest in sequence

From my test run #1 today I have written: The ultimate goal is to show interest in sequences that are unexpected or rewarding in some way and to repeat them until they are expected or lose their interest. The redo interest of LTM matches is used later after attention is paid to decide whether to start doing or to perform a reflexive response. Maybe it should be used in the attention paying decision.

The S-Habit#4 stimulus had sequential interest but lost its redo interest because it satisfied the 3rd active habit, which expected the 4.  This redo interest looks backwards and answers the question Have I experienced this before? Can attention be attracted to a stimulus that has been experienced before (was expected) but has a potentially interesting goal in the future? Provided there is no habit being concentrated on, then yes. It is also at this low concentration level that thoughts can attract attention!

What I need are “starting interests” for all experienced stimuli that are used to determine which stimulus to pay attention to if not concentrating and no stimuli have a redo interest (usually because they were expected). Is this akin to free association?

I tried this in Adaptron and it did not match up with the best stimuli to pay attention to. However the one test case #6 position 86 that is not working properly would work if it uses the following logic. When all the active habits are at neutral concentration and one of them completes (not necessarily the 1st one in the list) then this is the one that determines which stimulus to pay attention to.

21st Sept 2007                     A-Sequence execution

A-Sequences such as the R1-S2-R2 in the A-Habit sequence S1 R1-S2-R2 S3 form automatically when:

  • S2-R2 is about to be stored / performed by repeating a past S2-R2 experience and
  • the previous LTM is stimulus S1 with response R1 and
  • the S1 R1 S2 A-habit has been done previous to the past S2-R2 experience.

Such A-Sequences may result in the same goal stimulus S3 and have a neutral redo interest or result in an S4 goal and have an interesting redo interest. They may be redone because of the redo interest or because of thinking about S3 and its subsequent possible redo interest. When such an A-Sequence gets performed it may or may not be at an interesting concentration level. As humans we perform such sequences all the time. They are started consciously because of the desirability of the goal stimulus and then they are continued / performed subconsciously. They stop as soon as they fail to detect the S2 intermediate stimuli. While they are being done we can pay attention to something totally different, even thoughts, or we can pay attention to the stream of stimuli that the A-Sequence execution is producing. These stimuli do not attract attention because being expected by the A-Sequence reduces their interest level. We can also intervene in the sequence and perform a modified response instead of the A-Sequence one.

Since an A-Sequence A-Habit is started consciously but is done subconsciously it cannot contribute to learning. This means that it should not have a redo interest and should not form if an S4 goal stimulus is perceived. But then how does an A-Sequence A-Habit get combined with another A-Habit to form an even longer A-Sequence. The previous A-Sequence has to be concentrated on – conscious and laid down in memory and then followed by the A-Habit. Maybe an A-Sequence A-Habit that has neutral redo interest (has been learnt) is done subconsciously – it would have to be started due to thinking. While an A-Sequence A-Habit that has an interesting redo interest gets done consciously and stored in LTM for possible combination into longer A-Sequences.

28th Sept 2007                     Object Interest

As thought of on 17th July 2007 I have removed interest levels for primitive and parallel objects and implemented redo interest and expected interest in S-Habits and A-Habits in LTM. And it works. I have also been able to remove the property of last response from primitive and parallel objects and used LTM last responses instead. I should also be able to remove the permanent status property from these objects and determine it dynamically from LTM. Then the primitive and parallel objects will form a growing framework upon which “readings” are experienced. These “readings” will only be stored in LTM. They have interest, last responses and permanent status not the objects.

30th Sept 2007                     Amount of Change

When two sensors change by the same amount they get associated and recognized as the same object. But what if one changes by a negative amount and the other by a positive amount but both of the same absolute value? Are the two associated? I think so. The combination of the two that changed by the same absolute amount form a pair whose reading is determined from the two new values.

But we generalize before we specialize so I think that when a number of changes of different amounts all occur at the same time we capture the entire set of changed sensor readings as the experience. We don’t have to experience it multiple times each time removing those sensor readings which have unexpected values. With one occurrence we build a complete experience which can be used as the expectation.

6th October 2007                 Action Possibilities

An alternate strategy to performing actions is to first try all possible actions before practising them. This means that whenever a stimulus occurs and not all the possible actions have been tried for that stimulus then try the next action until they have all been tried once. This means not repeating past actions even if they resulted in interesting goal stimuli until all actions have been tried once. Once all possible actions have been tried then the most interesting of these experiences gets repeated. If two past experiences are of equal interest then do the most recently first. Keep doing this until all past experiences involving action responses for the particular trigger stimulus have a neutral interest in being redone. That means no more exploration for this stimulus and it becomes permanent and S-Habits can start to be formed.

                                                Redo Interest

When it comes to determining the redo interest of a habit it has a redo interest as long as there is a difference in the experienced change from the expected change. The absolute amount of difference between the experienced and the expected changes gives the redo interest level. The redo interest is either zero – neutral or a positive amount. There is no such thing as a negative redo interest – similar to the concept that there is no such thing as degrees of boredom or uninteresting-ness.

                                                Problem solving - Thinking

Problem solving seems to require the ability to think backward through ones experiences. When one has a desired goal one has an expectation of that desired goal stimulus. This must match a past experience that is associated with the memory of the current situation stimulus in order to find a past experience that contains a response capable of achieving the goal in this situation. The first memory trace (experience) from a trigger & response & goal sequence will not have any expected goal in it. The second time it is done one could record trigger & expected goal & response & same goal as before. This would then have the expected goal tightly associated with the trigger. It could even form a recognition habit (parallel habit) because the expectation of the goal occurs while experiencing the trigger situation. Using the idea of generalization just the matching of the expected goal part of the recognition habit could “trigger” the experience being performed.

But does the expected goal need to be recorded, couldn’t one use a reverse / backwards in time sequence association from the goal back to the trigger. That would work if the goal achieved is what was expected but what if the goal achieved is not what was expected?  It would still work. Now you have a proven old experience that you will still repeat if its expected goal is desired and a new experience that you will try if its goal is desired. You have not stored the expected goal of the old experience in the new experience memory trace. Without the expectation of either the new or old goal in the given situation you will repeat the new experience rather than the older one because of the unexpected versus experienced goal that happened and the fact that just the situation part of the recognition habit can trigger the experience. If the new experience is repeated (however triggered) a second time and it gets the new expected goal it will lose its redo interest. Then only the desire for its goal will cause it to be repeated in the given situation. The expectation of the old experience’s goals will no longer trigger the old experience’s action.

7th October 2007                 Speed of responses versus stimuli

We experience stimuli at a much greater rate than we can respond. So do we only respond to sequences of stimuli that we recognize? How can we ever learn to respond to a single stimulus? Adaptron will start responding as fast as stimuli occur when there is only one sense. I am thinking that in a multi-sensory situation it will take longer for the world to become boring and responses to start to be performed. That is true if the environment changes without any actions and it may explain, why as humans, we do a lot of observing before we get bored. But if the environment does not change on its own Adaptron becomes bored very quickly because it forms parallel recognition habits from the combined sensory inputs.

11th October 2007               Expected and Attention

Currently the perceived stimuli are compared with the previous stimuli to identify those that have changed and by how much. P-Habits change only if both their parts change. Stimuli are then interesting if they have changed. If no habits are being done the largest combination of senses that have changed will attract attention as a P-Habit. Then the S and A-habits are done and expected stimuli lose their interest if they caused the same amount of change as expected. Any stimuli remaining that are interesting above the concentration level attract attention. Else we pay attention to the habit expected stimulus. It is possible that when S and A-Habits are done the expected stimuli lose their interest just because they are found and don’t have to cause the same amount of change. Maybe I am confusing myself because there is the object expected and the reading expected. An S or A-Habit matches the expected object and has an expected reading. If the perceived object’s reading is different from the expected reading then the difference in readings probably determines the interest in the stimulus.

13th October 2007               Permanent Stimuli

Is it the object that becomes permanent or the object at a particular reading that becomes permanent? If it is the later then LTM experience should keep track of its permanence rather than the object. Thus the most recent response in LTM should be used to detect this state.

23rd October 2007               Permanent Stimuli

I have decided that it is the object at a particular reading / intensity that becomes permanent. I have also decided that a particular sense, sensor combination and intensity values forms the object and is different from a stimulus (object) at the same sense, sensor combination but different intensity values. I have also decided that LTM only keeps the objects experienced to form S-Habits. And therefore when an object becomes permanent it is a property of the object.

I have also changed the software such that if the habit being concentrated on fails to complete or fails to continue, then any other habit that completes or is continuing becomes the expected one. If one does not exist at this concentration level then reduce the level and find any habit that completes or is continuing. Keep on reducing the level until one is found to complete or continue. If the concentration level reaches zero and no habit is found then there is no habit expected stimulus.

24th Oct 2007                       Generalization and Specialization

From 4th Dec 2005 we have a definition of generalization. It occurs when a stimulus is a partial match to a multi-sensor or multi-sense trigger stimulus. We perform the past experienced action because it was rewarded. For example we see a particular book, open it and find it interesting because it has pictures. We see a different book, different size, different colour but it has some characteristics in common with the first one and therefore we perform the same action on the second book. If once again we get interesting pictures this experience is also worth redoing. Now any characteristic of the first or second book will trigger the same action. This can repeat many times until all books in general become interesting. Discrimination or specialization kicks in when we come across a book, open it and don’t find an interesting picture inside. How does this trial or repeated trials like this result in our recognizing which characteristic is necessary to specialize between the two kinds of books?

Let’s look at a simplified version. On the 1st trial the simultaneous stimuli A and B are followed by action 1 and the resulting goal stimulus is C which is interesting / unexpected. On the next trial simultaneous stimuli A and D cause the repeat of action 1 due to the partial match of the 'A'. The resulting goal stimulus is E which is interesting. On the next trial simultaneous stimuli F and B cause the repeat of action 1 due to the partial match of the B. The resulting stimulus is G which is not interesting. Now anything combined with B (except A?) will not result in the action 1. It appears the redo interest is associated with both the simultaneous trigger stimuli. Is this the same conclusion as the 31st March 2007?

25th Oct 2007                       Generalization and Specialization / Discrimination

If multiple trigger, action and goal stimulus sequences take place then each will either increase or reduce the interest in the individual stimuli that comprise the trigger. Thus some individual stimuli will become more than neutral interest while others will on average be of neutral interest and yet others become negatively interesting. But I am now attributing interest to a stimulus that I have previously decided should be attributed to the sequence as a redo interest.

14th Nov 2007                      Multi-sensor stimuli versus P-Habits

I have been storing the multi-sensor discrete stimuli as objects where the combination of sensors involved in the stimulus are represented as a string e.g. “001100101” where each “1” indicates that sensor’s reading is part of the stimulus. The resulting behaviour should be the same as for multi-sense P-Habit recognition where I capture a hierarchy of simultaneous binons. I have it working up until stimuli become permanent and S-Habits need to be formed. Test cases #20 and #25 or #26 and #30 are good examples. But placing background S-Habits on the habit stack in the right order, or processing them in the right order, or identifying the expected stimulus is troublesome for the multi-sensor data structure representation. So it is time to review the functionality required for processing multi-sensor stimuli and see if a better data structure / algorithm is possible.

A multi-sensor stimulus such as ABC represents 3 sensors where the 1st has a reading of A, 2nd B and the 3rd C. The readings are discrete and the three sensors are independent. An example might be each sensor detects the position of a different motor, each of which is controlling a different limb. The motors give off 4 possible position readings of A, B, C and D. When the ABC arrives it is compared with the last stimuli from these three sensors and those sensors that are different are flagged as such. Let us say the previous stimulus was ABB then the difference vector is “001”. The stimulus that is then found or added to the recognized list is “- - C” using sensors “001”. If the previous stimulus was ABC then the difference vector is “000”. The stimulus that is then found or added to the recognized list is ABC using sensors “111”.

The difference vector is then used as the unexpected vector that has its non-zero entries reduced by sequential habit execution. The remaining unexpected vector contains a “1” for any sensor that has a changed stimulus that was not expected.  For example, AAA is followed by ABC. The difference vector is “011” and the stimulus recognized is “- BC”. Sequential habits are expecting “- - C” and this reduces the unexpected vector to “010”.  If a stimulus has an unexpected value higher than the concentration level then the unexpected parts of that stimulus forms the new stimulus that is paid attention to. In the example if the concentration level is zero then “-B-“ is the unexpected stimulus attended to.

If however the concentration level is higher than any of the unexpected stimuli then we pay attention to the stimulus we would have expected based on the current conscious habit. When we attend to the expected stimulus and find that it is not what was expected the stimulus obtained instead becomes the conscious one attended to. Currently the expected stimulus is obtained from the expected stimulus of the habits based on the habit with the highest concentration level and the most sensors in the expected stimulus. This works fine if the concentration level is above neutral. However when the concentration level is neutral and lots of background habits are being done, nothing in particular is being concentrated on. If all stimuli have their unexpected vector reduced to neutral and no particular stimulus is being expected then it is reasonable to use the largest stimulus possible.

18th Nov, 2007                     Conscious Habit

I have found it is not important to keep track of which habit each stimulus completed. I was doing this so that when attention was paid to a particular stimulus I could identify the habit and which trigger stimulus in LTM needed its redo interest reduced. It now works because the expected stimulus is based on the currently conscious habit. If any unexpected stimulus occurs it is either more interesting than the current habit (concentration level) or the current habit does not get its expected goal stimulus. In both cases the current habit stops. If the current habit gets its expected stimulus then it’s the current habit’s trigger stimulus redo interest in LTM that needs updating.

However there still appears to be a need to keep track if a permanent stimulus completed a habit that already existed in LTM because if any habits are started with this permanent stimulus as trigger it will not be placed in LTM. When and if it becomes context that needs to be stored in LTM I need to know whether it completed a previous LTM habit or not and what the redo interest should be. But if this permanent stimulus did not complete a previous LTM habit, is it novel and would not be started, more likely to attract attention?

19th Nov, 2007                     Unexpected Stimuli

I have found that a simple flag on all just experienced stimuli that says whether it was expected by any active habits (or not) can be used to flag permanent stimuli. Then when they are kept as context and subsequently get stored in LTM the sequence they form is interesting (novel) or not.

                                                Not concentrating

If we are not concentrating, have not performed a response and have several background (permanent triggered) sequential habits expecting their goals and two or more get their goals, which one do we pay attention to? Two or more sequences are recognized and they are all of the same interest. For example we are watching a magician with a rabbit and a hat. The rabbit jumps into the hat and the magician waves his wand as expected. Both sequences have happened previously to the point where you no longer react to the individual stimuli but now form new sequences (S-habit results). Which one do we pay attention to? The best criteria might be to use recency. Either pay attention to the one that has a goal stimulus closest to where we were previously paying attention or pay attention to the sequence that has occurred more recently. I prefer the second approach, it’s certainly easier to program.

23rd Nov, 2007                     Action Possibilities

On 6th Oct 2007 I decided that all actions should be tried once in a given situation before any get repeated no matter how interesting the consequences. Then actions are repeated based on their redo interest. This is the nature of exploration. But if the first time an action is done it is rewarded with a pleasurable consequence then given the same situation further possible actions are not done, the rewarded action is repeated. Once this action becomes non-rewarding / not pleasurable then further possible actions are done based on interest.

24th Nov, 2007                     Action learning

I have been using a recall algorithm that finds the most recent LTM usage of an input stimulus to determine if there is any interest in repeating it. It effectively ignores all other past experiences and says the most recent experience is how the world operates now. Thus when it comes to repeating actions it tries the most recent action if it was worth redoing else it tries the next reflexive action. However with the changed strategy of trying each action once before repeating any after going through them all, the last action is the most recent. Based on its redo interest it is repeated and it then holds no more redo interest. Because it becomes the most recent experience none of the previous experiences with other responses even if they have a redo interest get done. It would seem that any past experiences with different responses with a redo interest should still be worth trying even though the most recent one is not.

To make this work the recall algorithm would find any LTM usage of an input stimulus that still had a redo interest. This would be the one worth repeating. If it succeeds then its redo interest would become neutral because it has lost its unexpectedness. If it fails and something else is obtained then the new experience becomes worth repeating. Then to stop the old experience from continuing to be tried after the new experience has lost its redo interest; the old experience’s redo interest must be neutralized. The fact that we continue to gamble on a slot machine seems to corroborate this algorithm. Even though the most recent experiences are unrewarding, even punishing because we lose the coin each try, as long as we have one past experience which was rewarded we continue to gamble. Mind you this example is dealing with pleasurable redo interest rather than exploration / novelty redo interest. But the recall algorithm should be the same. A failed repeat of a past experience should reduce the redo interest in that past experience. In the case of novelty / familiarity it is a single reduction to neutral where as in a pleasurable / punished situation it is a reduction but not to neutral.

Adaptron test run from 24th November 2007

25th Nov. 2007                     Stimulus obtained instead

On 11th April 2007 I wrote: “When a conscious habit fails then attention is paid to what was obtained instead or the difference between the expected and the actual stimulus.” I have been doing the second of these two possibilities for P-Habit recognition where the expectation is comprised of multiple parts. I have been paying attention to those parts of the actual stimulus that differ from the expected parts. However I now feel that it should just pay attention to the new combination of parts obtained as were in the expected stimulus. That is, it should pay attention to whatever was obtained instead even if some of the parts are different and other parts are the same. Otherwise when we expect to see a snake and we do see one we would only pay attention to those parts of the snake that differed from our snake expectation.

27th Nov. 2007                     Adding Graduated Senses

I’ve decided the next feature to add to Adaptron is graduated senses. I am using notes from 4th and 6th of July as guidance. The following note comes from the code. “Unexpected stimulus interest is based on the stimulus's change in intensity from the previous reading on the same sensor. Stimuli have no inherent interest. Stimuli may have some inherent pleasurable feeling though. Sequential interest is based on the similarity / difference to the last attended stimulus at LTM end and can only be determined after attention is reflexively paid to an unexpectedly interesting stimulus or intentionally paid to an expected stimulus by performing a past experience as a habit. Find the value of the biggest change experienced after sensory unexpectedness / change has been removed for some sensors by any and all habits being performed.” For graduated senses the unexpectedness of the stimuli on the sensors becomes of lower importance. It is the change in the object’s position, width and intensity that attract attention. Thus unexpected stimulus interest is not based on intensity change alone. Also it is a change in the object not in the sensors because the object becomes sensor position independent.

The current concentration level is really a level of the change in intensity that acts as a threshold. And this change in intensity is the integer value of (Log2 +1) of the absolute value of the actual change. Stimuli that cause a change in intensity above this threshold can distract attention if not removed by active habits. Concentration level will have to be augmented with a change in position threshold and a change in width threshold for graduated senses.

28th Nov. 2007                     Stimulus resolution

When you are presented with two stimuli at the same time you can detect very fine differences between the two. It may be two colour samples beside each other, two tones, of different frequencies, one after the other or two weights, one in each hand. Obviously according to Weber’s Law the resolution gets bigger the larger the value of the intensity of the stimulus. You can easily tell the difference between a 1-gram and 2-gram weight but will not be able to tell the difference between 501 gram and 502 gram weight. This describes the differences that are detectable by the sensors from one stimulus to the next because we have to be conscious sequentially of the two stimuli to make the comparison.

When you are given a tone of one frequency and told to remember it and then the next day given another tone of a slightly different frequency and asked if it is the same or different from the first the resolution is much worse than two sequential tones. This is because you are comparing a memorized stimulus with an immediate stimulus. The memorized value has been allocated to a bucket based on the Log2 of the actual stimulus value. This is true for all memorized values and makes comparison of experienced stimuli with memorized stimuli less accurate. (See comment 19th June 2007.)

This also applies to comparison of changes / differences. If you are given two tones sequentially and asked to compare the difference between them with the difference between another two tones the resolution of the differences follows Weber’s law. It is far more accurate than having to memorize the first change and compare it with the second change a day later.

Since real world values such as volume in decibels or brightness / luminance are measured over many orders of magnitude a logarithmic scale can be used to represent these values. Adaptron is designed to accept 256 values for each sensor reading. This means that the measurements taken by the real world sensors need to be converted into logarithmic values for input into Adaptron. This will mean that when Adaptron compares two sequentially perceived stimuli values as long as they have different logarithmic values then they will be recognized as different. But Adaptron will perceive two real world sensor values that are different but produce the same logarithmic value as the same. This would satisfy the Weber’s law situation.

But the fact that the intensity of a memorized stimulus cannot be compared with the intensity of a perceived stimulus with the same resolution as two perceived stimuli would seem to imply that a further logarithmic function is applied before storage. Or there is something in the comparison mechanism that uses less resolution.

I need to recognize those dimensions of stimuli which are pure intensity related, whose real world values cover many orders of magnitude and those dimensions of stimuli whose order of magnitude come from the multi-sensor array being used. For example volume is pure intensity while frequency comes from an array of sensors. Brightness is a pure intensity value while the values of position; width and stereoscopically determined distance result from arrays of sensors. The sense of touch, pressure and temperature are intensity readings while position on the body comes from the array of sensors. Weight is an intensity value while angle of a joint is I’m not sure. For those stimuli values that are determined by the sensory array I need to explicitly apply the bucketing logarithmic function before using their values.

29th Nov. 2007                     Sensor resolution

Maybe the simultaneous sensor comparison accuracy is a property of the sensor itself and its inability to measure more intense stimuli to the same resolution as the less intense stimuli. Then the delayed memory comparison is due to the bucketing of the intensity before it is recorded. This is what I concluded on the 19th June 2007.  So if we just consider intensity then a sensor measures it to what I am calling its greatest accuracy – its absolute value or raw value. Then it retains this value and can compare it with the next absolute value reading. It then produces a difference between the two readings based on these absolute values to greatest degree of accuracy it can based on its resolution. When the two readings and the difference get recorded bucketed values are used. Thus a first intensity of 128 and a second intensity of 32 get recorded as 8 and 6 respectively. The difference of 96 gets recorded as 8.

In the comparison of two changes given sequentially I suspect that comparing the difference between tone 1 and 2 with tones 3 and 4 is very difficult. This is because there is an intermediate difference between tone 2 and 3 that destroys the sensory memory of the difference between tone 1 and 2. This kind of comparison of changes probably works better with three tones producing 2 changes.

                                                Change as a Stimulus

Which brings me to an old question: Is a change also a stimulus and treated like any other stimulus or is it only an event / piece of information used to guide behaviour? It is definitely used to guide behaviour because it is used to determine unexpectedness. But can we pay attention to it alone, remember it and use it as the trigger of a habit? The fact that we can talk about it, as a thing would seem to imply it is a stimulus. The fact that we can compare two differences also makes me think it is a stimulus. And because behaviour can be triggered by a change independent of the intensity reading also makes it feel like a stimulus. For example you can ask someone to let you know whenever a tone changes its volume. Even more evidence that change is a stimulus is that we can recognize a tune such as Happy Birthday in any pitch and at a variety of speeds. See 18th Dec. 2006.

This leads to other questions, such as: Do we explore the domain of changes like we do a stimulus space? In other words, do we try all possible responses for all change stimuli? Then do we combine change stimuli with the intensity stimuli to get composite stimuli like we do with two sensors or senses? We appear to be able to use change as a stimulus for generalization because we realize we can get the same amount of change in a variety of situations. But doesn’t an S-habit represent a certain change. Maybe the S-Habit is a composite of the 1st and 2nd stimulus plus a certain change. It’s identical to the sensory array dimension, a 1st and 2nd position providing a width. Maybe an S-Habit can consist of 2 sequential changes and that is how Happy Birthday is recognized. See 18th Nov. 2006.

30th Nov. 2007                     Change as a Stimulus

Change is a property of sequence – not of stimuli. Thus it is only when a sequence produces an unexpected change that we pay attention to the change as a stimulus. Then the change can be recalled and a next change expected. Does the change have a sense? No. Because we can compare the direction of change across senses and the general magnitude of the change across senses. This would imply that change is its own sense for attention, recall etc. See 29th March 2007.

How can an unexpected change occur but still get the expected goal stimulus? Let us consider a graduated sense such as a range finder that is measuring 128. A change of –1 occurs and it is now reading 127. Memory records a distance of 8 in both situations. If later a distance of 128 is perceived and then another of 128 we have the same sequence, 2 x 8’s, but a different change. This can only occur if the environment changes without Adaptron doing anything. Another possibility is that a change attracts attention as long as the sensor detects a change but there is no difference in the bucketed values. This is provided attention has been paid to the same sense two times sequentially. Naturally this can only happen with a graduated sense or a graduated sensor.

11th Dec 2007                      Short Term Memory

I have been experimenting with the recording of bucketed values for graduated stimuli and changes as detected by the sensors. It is difficult to devise an algorithm that remembers the sequence of bucketed values and the changes without obtaining repeated stimuli plus changes. E.g. input stimuli of 8, 7, 6, and 5 are stored as 4, 4, 4, 4 and the last three 4’s each have a change of –1. Then I realized that maybe the short term memory keeps non-bucketed values of immediate stimuli so that sequences can be formed accurately and recognized but they get stored long term in degraded / bucketed values.

But how else might STM differ from LTM? What criteria are used to move stimuli from STM to LTM? Is it based on a max length, say 10, plus certain events that flush STM? Are there certain stimuli that do not get moved to LTM? Why not make LTM as accurate as STM and do away with bucketed values? This would be an easy thing to do until a capacity or retrieval limitation is found for LTM that is solved by bucketed values. But bucketed values in LTM give you a form of generalization such that numerous raw stimuli of different values match the same bucketed values. But this is also achievable during the match - recall process and still store raw readings.

It would seem that when short-term memories are bucketed and transferred to LTM the changes are more accurately stored than the raw stimuli values, thus allowing for the structure of the sequential pattern to be retained. Thus if raw values are stored long-term then recall must also find sequences of changes and match them.

                                                Generalization

Is generalization any use in exploration? Each situation is currently explored with every possible response. A situation such as one position in a maze is only seen as similar to another if the sensors are such that they detect the same pattern of stimuli. E.g. standing in front of a wall is the same situation as in front of any other wall if a range finder is used to measure distance to the wall. Can generalization be used to save time exploring a dead end corridor before going all the way to the end? Adaptron currently works this way with just 4 range finders and corridors with equal length because it cannot detect turns at the end of the corridor until it gets to the end.

                                                Sequence of Changes

So I have a need to recognize a pattern of changes. But do I need to explore a series of changes by trying all responses to such patterns of changes? A single change is a property of a sequence of graduated stimuli from the same sense.

12th Dec 2007                      Change and Objects

I need to go back to my thinking of 4th and 6th July 2007. Each graduated sensor with a discrete sense (independent sensors) continues to “see” the same object just with varying intensity. S-habits should be represented just as the recognition of two adjacent lines except sequentially. I had lowest level objects, a single stimulus, with an intensity of zero and then combined two adjacent objects with a relative intensity property to form a combination. This combination then had an intensity reading being the intensity of the first object. Translating this into sequential recognition means that in LTM I need to store the same object ID plus its intensity reading for primitive stimuli. Each S-Habit formed from two primitive stimuli has:

  • an object ID (its LTM position),
  • references to the two primitive objects at the LTM position and position + 1 plus,
  • the relative intensity of the two which is the change between their intensity readings.

Then the reference to this S-Habit needs the intensity reading of the 1st of the two primitive stimuli that matched the S-Habit pattern.

E.G. memory locations 10 and 11 contain primitive graduated readings of 5 and 3 and both have an object ID = #2. At location 10 is also stored the change / relative intensity between the two which = -2. The S-Habit #10 is captured at LTM position 10 as two object IDs = #2 with a relative intensity of –2.  At LTM location 45 I observe 2 primitive stimuli readings of 9 and 7. They match the S-Habit #10’s Object Ids and relative intensity so S-Habit #10 gets stored at location 45 with an intensity of 9.

14th Dec 2007                      Graduated Intensity and Objects

As a result of the separation of the object and its intensity / reading for graduated sensors it is now important to determine when, where and whether the reading and object is relevant or just the object. For example when recalling a stimulus it should match all objects independent of the reading but to determine the last response tried it should be based on the object and reading. Thus we explore the entire reading space of the same object. But when it comes to recognizing a sequence it’s two objects and their relative readings that is important and the sequence then has a reading. Thus the same sequence (of two objects) can exist with different readings.

20th Dec 2007                      Expected Sequence

So after recognizing the Happy Birthday sequence at two different pitches we then start hearing the same sequence at a new pitch. After the first few notes, all of which are individually familiar, we have established that it is the tune and we are then expecting the next note based on the change expected. Or we are performing the sequence expecting the next change; not necessarily the actual note. So I need to have a habit executing expecting the change! Then when the expected change does not occur, our interest is peeked due to the unexpected-ness. When the expected change does occur what we do depends on if the sequence is permanent or not. This would seem to imply that the sequence of changes becomes permanent rather than the sequence at a given pitch, thus contradicting 14th Dec 2007.

There is something to do with “getting used to” it. If every time we hear the tune it is at a different pitch we can’t “get used to it” where as if we always hear it at a certain pitch we “get used to” it at this pitch. A different pitch is novel until we “get used to” that pitch. If it is at a different pitch each time we “get used to” the novelty of the pitch. That is: there is no change in the change?

21st Dec 2007                      Get Used To

“Getting used to,” means that you have a habit that expects the stimulus and causes its unexpected-ness to be neutralized whenever it happens. Let us go back to the 1st paragraph of 20th Dec. If we continue to hear the tune at different pitches the first note will produce an interest with no particular change expected. But the second note produces an expected change based on the 1st change between the 1st and 2nd notes. If the 3rd note confirms the 2nd expected change then the unexpected feeling is neutralized even though the actual values in this sequence have not been heard before.

Following on from 11th Oct 2007; when it comes to changes in general it starts with the sensors detecting a stimulus with a value and a change in the value from the previous reading. If there is no change the stimulus is not interesting and does not interrupt attention. Then the active habits are expecting certain possible changes from that sensor. If any one of these habits gets its expected change matching the sensors actual change the stimulus loses its interest and does not interrupt attention. If a stimulus produces an unexpected change then the stimulus is interesting and may attract attention. If it is attended to it becomes the conscious stimulus. This stimulus now has a sequential change from the previous one attended to. Its sequential change is used as its redo interest if its interest was neutral. If there is no change then we are consciously bored. If the attended to stimulus and the previous one are from different sensors / senses we only have sequential change. If they are from the same sensors / sense then we have conscious change in value which should be used as the sequential change and the redo interest. But the sequential change should be reduced by any conscious habit being performed that expected this change thus making the sequence boring. It seems that before the stimulus becomes the attended to one background habits (neutral interest) are reducing the interest of stimuli. After the stimulus becomes the attended to one the conscious habits are reducing the sequential change interest.

So given a first note, there may be a number of expectations produced as a result of experience. The sense / sensors themselves expect the note and value to repeat. Based on a stimulus’s object and value we could have some expected next stimulus objects with values on the same or different sense / sensors. Plus based on its object alone we could have some expected changes in value on the same sense / sensors. The first expectations are sensory habituation. The second expectations are LTM S-Habits. The 3rd are conscious STM attended to Change Habits (C-Habits). They are used to recognize or expect sequences independent of the values.

When the same tune is repeated at different pitches the S-habits never get their expected goals. However the C-Habits get their expected changes. With S-Habits if one succeeds we reduce the redo interest to neutral and the next time (3rd time) the trigger occurs we have no redo interest and do a reflexive response. If all S-Habits fail the redo interest remains. What do we do if a C-Habit succeeds? Does this reduce the interest after the S-habits have done their thing? This would mean that after the second time the tune is heard even though it is at a different pitch from the first time it (the tune) would become boring and the 3rd occurrence would trigger a reflexive response. Is it the tune (sequence of changes) that is boring or the actual experience with the pitch values that is boring? In this situation it is the tune that is boring. When an S-Habit succeeds the experience becomes boring. The same is true for a combination of objects that occur in parallel. Different combinations of familiar objects are interesting. Modern art sculptures made up of parts of everyday items are good examples.