2011
4th Jan 2011 Remain Interesting
It is possible that a concept remains interesting even though specialized instances lose their interest when they occur for the second time. Books are interesting but once read any particular book is no longer interesting. The concept of "book" is the aggregation of the common properties that all books have. When we are only dealing with concrete / symbolic stimuli there are no common properties. However for P-Habits there are common parts. But without these common parts can we have the thought of a specific concrete symbolic stimulus. We certainly recall them, but do we create a thought from this recall? In reality this must be a thought of a particular object and a singular object like the Eiffel tower would serve as an example. The thought of the Eiffel tower in my mind takes the form of an image that I have of it from my experiences. There are also many other associated images of it from different directions.
Storing thoughts
We must be able to store thoughts and recall thoughts because we can recognize a thought / idea when we have it again. When we aggregate common properties to create an idea / concept we can associate it with a word, store it and recall it. The idea created is not an exact match to any particular experience but found as part of particular experiences. The same applies to any imaginary thing built from experienced parts such as a dog with the head of a parrot. So this changes the ideas that I had on 15th Dec 2010. When one has a thought it can match with a stored thought as well as a stored perception.
5th Jan 2011 Concentration
Concentration is just the attraction of attention and the holding of attention on a particular subject or chain of thoughts / line of reasoning. (1st July 1999) Thus attention is attracted to thoughts that have an expected interest or expected value. Just as experience assigns/reassigns interest or value to perceptions so does thinking assign/reassign expected interest or expected value to thoughts.
Kinesthetic feedback
I have a number of reasons to believe that kinesthetic feedback stimuli are needed as part of the goal of A-Habits for recognition and selection. First assume there has been one occurrence of the A-Habit AxB and B is interesting. Then assume the next action sequence triggered by A becomes AxC. AxC is the most recent action habit and must replace AxB for selection purposes independent of what interest there is in B. The recognition of which action habit AxC replaces is based on the A and x, not just the A trigger.
What if the world changes and AxM happens and M is already familiar and of neutral interest. It will be the most recent and replace AxC. Will we have any incentive to try x again in situation A. The action habit AxM is novel but not the goal M. I believe the answer is the goal is the aggregation of the stimulus M and the kinesthetic feedback stimulus X from the x response. And even though both M and X are uninteresting the combination is novel and interesting.
While in the womb I believe we build up the links between responses and resulting kinesthetic feedback stimuli. There are no other sensory stimuli other than the kinesthetic ones and the sense of touch. And touch only works when it is self-touch. In this environment we build up action sequences (R-S-R) and parallel actions and recognize S-Habits of kinesthetic stimuli that are the Ss in the R-S-Rs. By the time we are born most of the possible S-habits are known and are familiar / uninteresting. And given any current limb position Q and a desired limb position P as an idea Pe we have the action sequences / habits in the form Q_Pe and Pe r P to get there.
6th Jan 2011 Action Sequences
The list of possible actions starts out with just the primitive responses. As action sequences are created they are added to this list of actions. When Adaptron needs a new reflexive response it takes the next one from the list of primitive responses. If it were to instead take the next one from the list of actions without considering whether it was primitive or not it would have an ever-growing list to select from. If it were to reach the end it could start over at the beginning again. In this way stimuli would never become permanent. Also the non-primitive action sequences on the action list would be sequences which have been successful in the past.
Kinesthetic feedback
Another possible scenario that may require kinesthetic feedback is one in which AxB has occurred twice has been remembered and B is uninteresting. Now AyB occurs getting the same goal. Currently Adaptron replaces the x response in AxB with the y response. Thus no memory of doing x is kept. The most recent way in which it accomplishes the goal is the way it will do it again if the idea of B becomes interesting.
Goal / Objective trigger
When I 'go to the kitchen' for a glass of water I carry with me the trigger stimulus that initiated the thought about plan and course of action. It was the feeling of thirst. It gets quenched when the goal is reached. When I go to the ATM to get some cash I must also carry the trigger stimulus with me, the need for cash. However in this case it is not physically present to remind me so I must set it up as a thought. And should I get an alternate thought of a need while going to the ATM I could be distracted. This is less likely when fetching a glass of water.
Learning to think
If thinking is to be learnt it needs a process like the one used to learn to act. The thoughts may be generated continuously as stimuli are attended to but attention must be attracted to thoughts, mental orienting responses (as of 3rd Oct 2010) reflexively performed and then later given a similar trigger thought they must be done again if there is an interesting goal. All the time that this happens memory of stimuli and responses must be treated as the environment upon which thinking is acting / experimenting. The automatic generation of thoughts is easy to envisage. It is just the recall of remembered stimuli. The performance of thought about actions is not so easy to envisage. It seems to be the feeling we get when we will / decide to proceed and do something. It is the point at which the chosen Acton is started.
7th Jan 2011 William James
From his second volume chapter entitled "Will" he explains the following. First, it is the anticipation of the kinesthetic effects of the movement that is represented in consciousness. Second, if a person forms a clear image of a particular action, that action tends to occur. The occurrence may be inhibited, limited to covert tensions in the muscles, but in many cases having an idea of an action is sufficient for action.
Learning to think
Following on from the above subject, thoughts that are generated continuously are currently the ideas / expectations of goals associated with the attended to stimuli as triggers as recorded in action habits. A stimulus / perception does not generate / elicit its own idea. For attention to be attracted to thoughts / ideas they must have some attention attracting property. Currently this is the expectation of interest. The currently perceived stimulus must have already been conscious, recorded and recalled to match the trigger of an action habit. The mental orienting response to the idea is the process of it becoming conscious. The only action one can do mentally is to pay attention to the recall of a recorded perception or idea. However, since perceptions are aggregation of properties and could be sequential as well, the pay attention action may be able to select part of a memorized perception. Also we can pay attention to the next independent stimulus or one based on a particular Acton such as what is behind you. This last mental orienting response produces the idea of a goal based on the response part of the action habit. But if Actons are to be subconscious, output only, then the selection of the idea of a goal from all the goals of all the action habits that are associated with the trigger must be based on a property of the goal. And this is where the goal must be a two-part stimulus. There must be the external world perception part and the kinesthetic part. For answering the question "What is behind you?" we learn to select the goal that has the appropriate kinesthetic part for "turn around".
This "turn around" kinesthetic stimulus must be a sequence of kinesthetic stimuli rather than the final body position. The final body position would not be adequate to identify the action that got you to this position. However during the performance of the "turn around" action a series / sequence of primitive responses were output and the sequence of feedback kinesthetic stimuli were received. This would seem to imply that if two action habits were done in series the two kinesthetic stimuli are combined into an S-Habit as part of the goal even though they may be independent. For example we have the action habits AxX and XyY where x and y are primitive responses and they produce kinesthetic stimuli X and Y respectively. Because Y is interesting and X is uninteresting, thinking will cause us to perform and create the action habit A (xXy) Y. But the goal is not just Y it needs to be the sequence X,Y. And if we have an external sense as well producing B and C stimuli the action habits would be AxX:B and XyY:C. The combination might be A (xXy) (X,Y):C where X,Y is a sequence and C is instantaneous. Or is it A (xXy) (X,Y):(B,C). Maybe the X,Y and B,C are created in conscious STMs rather than the ones that create the independent sequential stimuli from the dependent ones. Thus conscious STM combines independent stimuli. We certainly have such a conscious STM for combining ideas such as when we combine images sequentially. (See 13th Dec 2010) But if x and y are the two primitive responses to do "turn around" how do we get just the C stimulus as the goal of the action habit? Maybe only the C is combined with the X,Y as the goal. Maybe the B is expected and familiar whereas the C is novel and remembered.
Consider a robot on a checkerboard of squares where a letter identifies each square and it can move in all 4 directions as primitive responses. Once it has explored the space, thinking should be able to answer the questions "What is to the left?" and "What is behind you?" The kinesthetic stimuli identify the mental orienting response to perform and the associated stimulus to pay attention to.
If we keep kinesthetic stimuli as part of the goal stimulus then the second time AxX:B gets performed it will start with the automatic thought habit A_Xe:Be before doing the x response.
Do non-kinesthetic stimuli serve any purpose as intermediate feedback stimuli in Actons? If they do they make the Acton situation specific and thus not very reusable. Thus I would suggest that Actons that contain only kinesthetic feedback stimuli are reused while the others will always fail when reused other than in specific circumstances. But what about non-kinesthetic stimuli such as generic properties rather than specialized situation being used to produce general purpose Actons? This should work and allow us to create Actons such as "get a glass of water".
So now let us consider the conscious STM creating a sequence of independent stimuli provided we have a current action habit being performed. It's like listening to the music you are playing without being conscious of the movements you are making [self-entertainment]. If you are in the womb the only feedback is kinesthetic and it would soon become uninteresting. However by combining action habits sequences of kinesthetic stimuli could be created. There would be a lot more variety in these sequences.
Loops
We must also be able to keep and repeat a looping action habit. In this situation the goal stimulus is the same as the trigger stimulus. To be able to do this we have to rethink permanent stimuli and what to do with sequences of independent stimuli formed in conscious STM. Maybe a loop action habit should be repeated until it has been shown to repeat twice in a row. This would allow a single loop action habit to be done. It would then get repeated again if some other sense produced an interesting change even though the kinesthetic feedback was the same as in the trigger.
8th Jan 2011 Mental orienting response
During thinking we recall experiences from memory based on all the same criteria / properties that we use to identify perceptions and by which we pay attention. But also during thinking we recall experiences from memory based on the response/acton that was done to obtain the stimulus. The retrieved experience is the one associated with the kinesthetic stimulus sequence that the responses produced. It seems reasonably easy to understand a kinesthetic sequence for "turn around" as a single object but it gets more difficult to understand a similar unitary object representing "go to the kitchen". The latter involves not just kinesthetic stimulus feedback but also visual. One must recognize the edges of the doorway to walk between them. This is most likely part of the walk action habit. Now "go to kitchen" is a little easier to understand. It may consist of "get out of chair" and "walk to kitchen". But this combination is going to be done when starting from a sitting position. A different combination must be done if one was lying down. It would be "get out of bed" and "walk to kitchen". The 1st Acton gets you into a state where the next can be performed.
But what about recalling experiences from memory where no action was performed. Looking at a movie is not a good example because eye motions feedback is used to recall things "beside" or "above" others. Consider listening without any head motion. Here the sequence of independent stimuli would be attended to and there would be no kinesthetic feedback stimulus. Or would "no motion" be the stimulus. What I have programmed as the orient response. I think so. This would give us the mental orienting response called "next".
There is also the recall of those novel images that you experienced as you were walking. They also feel like they are based on the "next" / "no motion" kinesthetic feedback stimulus. But they must be based on the "walk" / "go" feedback. It's like asking, "What did you see while walking?" You will recall the images in order, each one being the trigger for the next as the goal of "walk". Any familiar images experienced or sequences of familiar images must have been expected and you will not recall them from your walk because the walk action habit expected their change. But how does it eliminate all the change from a scene that is novel in its composition but familiar in all its parts. This is because both parts of a pair must change for the pair to have changed. If all the parts at the lowest level of a multilevel scene are all familiar and all their changes are familiar there are no unexpected changes. This is how we can disregard the trees in a forest even though everyone is unique and unfamiliar. A tree will only attract attention and be remembered when one or more of its properties are not neutralized by the expected properties of tree.
The use of the orient action stimulus for the no motion or next feedback stimulus may eliminate the need to create a conscious STM. If all thought sequences were kept as mental action habits what would be the feedback stimulus that we use when we chunk 7 plus or minus 2 ideas together? No, this will not work. We need to be able to combine thoughts in a tree.
9th Jan 2011 Interest
Change of parts is combined such that the largest combination of parts that have changed attracts attention. Does interest work the same way? I have been presuming it does. If only one part of a two-part object is interesting then I have been presuming the whole is not interesting. The assumption has been that both parts must be interesting for the whole to be interesting. Well if a goal stimulus is the combination of a kinesthetic stimulus and another sensory stimulus and the kinesthetic part is not interesting the goal would not be interesting. But interest is based only on novelty. That is, if it has not been experienced before it is novel and therefore interesting. This means that if one part is interesting it is new and when combined with any other part, new or not, the combination must be novel and therefore interesting. Thus interest does not follow the same rules as change. Change attracts attention when dealing with sensory stimuli. But for thinking and repeating action habits interest is used. Thus attention would be attracted to memories in which any part is interesting. This would mean a goal that consists of one neutral kinesthetic stimulus and one interesting other sensory stimulus is interesting. But is this correct? Can only one of its parts determine the value of an object? It is true for an apple in which the core has no value. It is true for a cage with a snake in it. You don't want the combination.
However on testing this using test case #20 the combined situation of / and A is experienced first before either / or A have been consciously experienced. By the time / or A are experienced consciously the / and A is familiar and already uninteresting. Does it become interesting again because one of its parts is interesting? Or do / and A become interesting even though subconscious and / and A only becomes uninteresting when both / and A have become conscious and uninteresting. I think this is more likely. Thus interest in any combination of stimuli is determined based on the interest of its parts and not on the number of times the combination has been experienced. However in the situation where A never changes but the kinesthetic stimulus changes from / to a and to b the A will always be interesting because it never becomes conscious by itself. Is it possible that attention is attracted to sensory stimuli that change as well as those that have value including interest? The value / interest attention would be attracted after the change attention attraction produces no stimulus (i.e. the whole scene was expected). This is true when you are in a crowded room surrounded by many conversations that you have tuned out because they are all expected. Then when you name occurs in one of the conversations your attention is attracted to it.
I need to introduce dependent combinations and independent combinations of parts or in other words the smallest interdependent combination. These have novelty, interest, and familiarity as a single unit. Their parts are all new until attended to separately.
10th Jan 2011 Independent Stimuli
When dealing with independent sensors on one sense or independent senses we must determine whether a composite stimulus is dependent on some other stimulus and therefore subconsciously recognized or whether it is independent and becomes conscious. If we have many senses each producing a symbolic stimulus then the 1st combination ever produced is dependent. An example is ABCDEFGH. All combinations of stimuli at all levels of complexity are dependent except the highest level combination of all 8 senses that must be independent since there are no other senses with which to be dependent. All stimuli start out dependent. If a subset of these stimuli changes in the next frame then the subset that did not change is an independent composite stimulus. An example is ABCDJKLM. ABCD is now independent, as is the highest level combination. All other combinations are dependent. A dependent combination stimulus such as ABCDJK can have a part that is independent. Having part that is independent does not make the whole independent. The idea that the combinations of values on the senses that remain the same become an independent stimulus is similar to the idea that in sequential recognition if a stimulus value repeats it is sequentially independent. When the next frame occurs it will have some senses that have changed and some that have not changed. A subset of the ones that have not changed may overlap with already independent objects. The biggest subset of the unchanged senses that are a subset of an already independent stimulus is now independent. This rule works on the 2nd frame because the whole of the 1st frame is independent by definition. That which remained the same is the biggest subset of the whole. An example for the third frame is ABNOPQRM. The AB is the biggest subset of the independent ABCD and therefore AB is independent. ABM is the biggest subset of the independent whole ABCDJKLM and therefore ABM is independent.
However it needs to be more general. If the next frame contains STUVWFGX everything changed but the FG is independent. So whether the stimuli stay the same or change from one frame to the next is not important. The more general rule is the biggest subset of stimuli in the new frame that is a subset of a known independent stimulus is independent.
Attention attraction
Any stimulus that changes does not necessarily attract attention. In the case of stimulus sequences the changes are used in subconscious STM to build up patterns of sequential stimuli and only when a repeat occurs and an independent sequential stimulus is recognized does a change become conscious and attended to. This is similar for dependent parallel stimuli. The dependent stimuli may change but not attract attention. It is only changes in the independent parallel stimuli that attract attention. Thus in test case #20 the /A, aA and bA are all independent as is A. Although /, a and b change they remain dependent stimuli and do not attract attention. However when they do change the combinations with A change and these are the independent stimuli that attract attention. This results in the identical response trace from test case #19 as should be expected.
12th Jan 2011 Independent P-habits
What is a bottom up algorithm that will identify independent P-Habits from the dependent ones? If we use the ABCD, ABED, XZLN, XHLP, XZCD, XBLD example sequence the following has to happen.
ABCD All lowest level symbols and all P-habits are novel and dependent except ABCD is novel and independent. It will be Interesting and Independent if attention is paid to it.
ABED On the 1st pass A, B, and D are known. E is novel and dependent.
On the 2nd pass AB, BD and AD are known. AE, BE, and ED are novel and dependent.
On the 3rd pass ABD is known. ABE, BED, and AED are novel and dependent.
On the 4th pass ABED is novel and independent.
XZLN All lowest level symbols and all P-Habits are novel and dependent except XZLN is novel and independent.
XHLP On the 1st pass X and L are known. H and P are novel and dependent.
On the 2nd pass XL is known. XH, XP, HL, HP, and LP are novel and dependent.
On the 3rd pass XHL, HLP, XHP and XLP are all novel and dependent.
On the 4th pass XHLP is novel and independent
XZCD On the 1st pass all the symbols are known.
On the 2nd pass XZ and CD are known. XC, XD, ZC, ZD are all novel and dependent.
On the 3rd pass XZC, XZD, XCD and ZCD are all novel and dependent
On the 4th pass XZCD is novel and independent.
However X is independent - it is known and is part of independent XL
And D is independent - it is known and is part of ABD
XBLD On the 1st pass all the symbols are known.
On the 2nd pass XL, XD, and BD are known. XB, BL and LD are all novel and dependent.
On the 3rd pass XBL, XBD, XLD and BLD are all novel and dependent
On the 4th pass XBLD is novel and independent.
From the ABED stimulus combination one might conclude that the largest known P-Habit (ABD) should be made independent. But in XZCD at the 2nd pass we have 2 two largest known P-habits (XZ and CD) both of which need to be made independent. So the rule could be all the known non-overlapping P-Habits at the highest level. This rule would not help make X and D independent. And it runs into a problem in the last XBLD stimulus combination. Here XL, XD and BD overlap in their use of the next level down's stimuli. A simpler rule of all the known P-Habits at the highest level will not work for a combination of ABCDEF where ABCD is the highest 4th level known P-Habit and DEF is the highest 3rd level known P-Habit. Each higher level known combination uses one or more of the lowest level symbols. We want to pick the highest level known P-Habits that use a lowest level symbol for the last time. So each lowest level symbol could be assigned the level number that uses it and the last level that used it would be on that symbol. No, try assigning a flag to the level below combinations that are incorporated in the next level up known combinations. Then any known combinations that are not so flagged must be the independent ones at that level. DE, EF and DF would be flagged in producing DEF but DEF would not have a flag. ABC, ABD, ACD, and BCD would be flagged in making ABCD but ABCD would not be flagged. But then the problem is how to flag the ones not actually used to form the combinations yet still part of the combination. The not used ones are known P-Habits that share one of the two parts used to form the next level higher combination. ABC and BCD are combined based on the common BC. ABD shares AB with ABC and ACD shares CD with BCD. This can be checked and they can be flagged. But it does not work for the next level down.
Maybe we have to go back to the idea that each known P-Habit is expecting only a certain number of left and right matches and soon as it gets an alternative match it must be independent.
13th Jan 2011 Independent P-Habits
I believe the solution is an algorithm that works top-down as well. Firstly every known P-Habit stimulus and its parts which may be dependent down to the bottom level will have a "Used in level number" (UL) which will be the level of the lowest independent stimulus they are currently part of. Brand new (previously unknown) symbols and P-Habits will always have their UL set to the highest number of senses i.e. 6. An independent P-Habit will have its UL set to the level it is at. ULs of dependent P-Habits will remain at 6. All P-Habits will also have a "Found at level number" (FL) that will be dynamically set as they are formed during the bottom up recognition process. Brand new symbols and P-habits will be dependent and have their FL set to 0. As known P-Habits are formed from the bottom up they and all their parts down to the bottom level will have their FL set to the known P-Habit's level. In the forming of ABC, ABD, ACD and BCD these will all have an FL=3 as will A, B, C and D. When ABCD is formed ABCD, ABC and BCD will have an FL=4 but ABD and ACD will remain at 3. ABCD and all its parts will have a UL=6 and still be dependent. Now coming down from the top of ABCDEF and looking at every P-Habit formed. We first find ABCDEF dependent with UL=6 and FL initialised to 0. Being at the top level it is changed to independent. Then at level 5 all the P-Habits are new so they get a UL=6 and a FL=0. They are all new so they are dependent. At level 4 we encounter the dependent P-Habit ABCD that is known with UL=6 but FL=4. You find the lowest FL value (4) for all the level 1 symbols used as part of ABCD and this is the new FL for ABCD. If this is higher that the level the P-Habit is at (4) then the P-Habit stays dependent because it is used in a higher level known P-Habit. But in this case it is not greater than 4 so it is the highest and it is made independent and its UL is set to 4. All other level 4 P-Habits are new so they are dependent and have UL=6 and FL=0. At level 3 ABC and BCD are dependent with UL=6 and FL=4. They have been used at a higher level so they remain dependent. There is no need to search their level-1 parts for the lowest FL value. However, ABD and ACD are dependent with UL=6 and FL=3. You find the lowest FL value (4) for all the level 1 symbols used as part of each P-Habit and this is the new FL for these P-Habits. If this is higher than the level of the P-Habit (3) then the P-Habit stays dependent because all of its parts have been used in a higher level P-Habit. Any independent stimulus on the way down remains independent with UL=its level and FL changed based on its usage on the way up. A stimulus with a FL greater than or equal to its UL must already be independent. How does this work when you find DEF at level 3 with UL=6 and FL=3. When you look at the FLs of its level 1 symbols you find D's FL=4, E and F's FL=3. Since you use the lowest value (3) the FL for DEF is 3 and DEF becomes independent with UL=3. DF at level 2 with UL=6 and FL=2 gets its FL increased to 3 because F's FL=3. FL=3 is greaater than DF at level 2 and it remains dependent.
This algorithm does not use the UL value to check that a known P-Habit is part of a smaller than the largest independent P-Habit. Even if we were to assign the UL value of an independent P-Habit to all its part's ULs this would not be unique enough to identify which independent P-Habit it is part of. What if we replaced the independent P-Habit's dependent parts' ULs with pointers to the independent P-Habit? Now as we form the known dependent parts we should only assign the FL if they all have the same pointer? No we may be able to not use the FLs at all. As we come top down the brand new P-Habits will be set to point to the top most P-Habit. When a known P-habit is encountered it will be set to independent if it is the first one with a pointer to a P-Habit at a higher level. We will change it to independent; make it point to itself and all its parts that were pointing at the higher level to point at it. If we encounter a known one that points to itself then we know it is independent already.
16th Jan 2011 Intensity
When listening to brass music last night it was the most relatively intense instrument that attracted my attention. It stood out amongst all the instruments. But when paying attention I could pick out a particular instrument and follow it based on its particular pattern of sound. I could also pick out a regular background beat.
Subconscious action
As I subconsciously perform well-practiced and learnt action habits they feel like dependent sequences of stimuli. Maybe we use our conscious STM to build ever more complex and interesting sequences of the stimuli generated by actions. As they become familiar as a result of paying attention to what we are doing we can then disregard the "what happens next"s as we do the action sequences. Maybe in this way we pursue ever more complex stimuli sequences that are interesting and don't need to have the concept of permanent stimuli. In sequences there is a never-ending possibility for novelty.
20th Jan 2011 Action Habits
I went off onto the dependent / independent analysis of P-Habit stimuli because I felt that the goal of an action habit must contain something that represents the response performed. This would be the kinesthetic part of the goal. And this would be combined with the visual or auditory etc. part. But now I think about the sequence of A/ and Aa etc. from test case #20 and I think that the action habit must contain in the goal stimulus those parts that have changed so that they can be expected when the action is performed. The 'A' part of the A/, Aa sequence is p-independent and does not change whereas the / and a kinesthetic parts do change. There should not be the need to store the 'A' part in the goal. This seems to imply that the changing part of the P-Habit attracts attention and is the part that is conscious. But my P-Habit dependent / independent analysis has / and a p-dependent on 'A'. This would seem to mean that / and a are independent because they changed independently of the 'A' and thus the stimuli that comprise a change are also p-independent. Or as long as the independence is in one direction then it is in the other one too. Thus / and a are p-independent of 'A' even though 'A' is the only non-kinesthetic stimulus that occurs at the same time.
The common coding theory is exactly what I am thinking about when I say the goal contains the kinesthetic stimuli generated by the action and thus is representing the response performed in an action habit. But when two lowest level action habits are strung together does the goal contain the stimulus sequence of two kinesthetic goals? If one has been observing one’s own actions (self-entertainment) then the sequence has been consciously observed. This would be in conscious STM and maybe starts being chunked. How do we stop stimuli in the sequence from initiating the lowest level action habit that is already in progress? And it does not feel like the goal contains the observed sequence of kinesthetic stimuli. The goal only contains a representation of the final state.
21st Jan 2011 Intentions
I have often said that if we have a word for it we must be conscious of it. This applies to intentions. An intention corresponds to a verb. An intention is the thing to be done that is going to achieve a desired goal. Thus it is the stimulus we experience that corresponds to the action we perform. This makes it the identifier of the response or action sequence in the middle of an action habit - i.e. the Acton. But not the one sent to the devices. It is the one that the cerebellum provides to identify the action sequence. This stimulus must make up part of a goal such that we can answer the question and imagine (think about) what is behind you or what happened next.
When we answer the question ‘what colour is it’?, ‘what shape is it’? and ‘what is it’?, we are asking for the values of the colour, shape and composites of such properties. Thus our attention is paid to these properties and the values returned. The property is the where information for the stimulus. Similarly the intention must be a where like property of a goal or action habit. It is a sequential property rather than a parallel one. When orienting responses are being done the property is 'next'.
Wikipedia however defines intention with the following sentence: An agent's intention in performing an action is his or her specific purpose in doing so, the end or goal that is aimed at, or intended to accomplish. But this is the intention of the act not the intention itself. It might be better to say an agent's intention is made up of two parts, the action and the goal.
22nd Jan 2011 Summary
I have the following changes to make.
- Do not have a permanent stimulus status - just use the action list repetitively including the non-primitive Actons.
- Use the Acton identifier in an action habit as a piece of where information for paying attention.
- Use the Acton identifier "next" as where information when no physical action is involved.
- Combine sequentially independent stimuli in sequences in conscious STMs per sense when the attention action is "next". This will produce novel sequences.
- When thinking pay attention to the goal based on the Acton identifier as the where information.
- Thinking performs the Acton identifier actions on the memory of experiences. Thus we have thought versions of the Acton identifiers that are the attention actions to be done while thinking.
- Need a thought about goal that comes back from memory as unknown because although it can be attended to (it has a where) it has not been experienced.
Attention
Attention is first paid to any unexpected changes. When there are no unexpected changes any stimuli that have an interest or reward attract attention. When there are no such interesting stimuli then any action habit associated goal with interest or reward attracts attention.
Failed action habits
I had one of those "what did I come here to do”? moments today. I was in my office and thought about eating a chewable vitamin C. I immediately started the action habit to go to the kitchen. But once I was there I could not remember why I was there. Most of the time being in the kitchen triggers the next action habit because you still have the desire. But there was no hunger involved so the kitchen as a trigger did not remind me of what I should do next. I had also failed to set up the reminder using the kitchen as the trigger. I had thought of the association from vitamin to kitchen and started the “go to kitchen” Acton. But I had failed to think about the second action habit from kitchen to open the cupboard and get out a vitamin C. Once in the kitchen I had a decision point. This is a good example of what is involved in a decision point.
24th Jan 2011 Subconscious Actions
Actions started by thoughts are done subconsciously. They are not re-recorded because attention is paid to the ‘next’ stimulus. Attention is freed up to notice whatever happens next or to think.
25th Jan 2011 Actons – Action Sequences
I realized that rather than starting an Acton (action sequence) when a habit is started I should have the Acton initially created by a first action habit when it completes. Then when the next (second) action habit completes if there is a previous Acton started with a feedback stimulus the same as the trigger for this second action habit then complete the Acton and make its identity available for LTM of the combined action habit – first and second.
Loops
Action habits with just a ‘next’ action should be used to ignore repeated independent stimuli. A looping acton in which the first action is the same as the second action should be repeated as long as the feedback stimulus continues to match the expected. Once I have actons that run in parallel I should be able to set up multiple subconscious action habits that loop so that constant familiar stimuli from multiple senses (such as the feel of sitting down and the hum of a fan) are expected and ignored. Then only changes out of the ordinary will attract attention.
29th Jan 2011 Looping
I should have the execution of an action habit repeat if it can. This would mean they repeat by default and only stop when they fail to get their intermediate feedback stimulus. An alternate definition of a looping acton could be one where the first and second actions are the same and the final goal stimulus that the action habit execution gets is the same as the intermediate feedback stimulus. Brings back memories of TOTE!
7th Feb 2011 Parallel actions
Actons must be able to be added together in parallel such that I can start by rubbing my tummy with my left hand and then patting my head with my right hand. Each of which is a looping acton. Then the combination must be doable as a single subconscious acton. This is used when an executing action gets you close to a goal and neutralises the stimuli changes it expects but there is still a little bit of unexpected stimuli remaining. You perform an action to eliminate this bit remaining and it gets added to the overall acton such that all possible minor variations are accounted for. Only then can the new acton / behaviour be performed without any conscious interruption. Also added parallel action habits must be in operation to disregard all the continuous background stimuli that result from the action. An example would be the noise of cars on the road as you are walking on the footpath or the feel of the seat you are sitting on when you are working at your desk.
Attention while acting
While an action sequence is being done subconsciously and it is neutralising stimuli changes, there are two things the conscious attention can do. It can think about past experiences or it can pay attention to the pattern of stimuli that are occurring. The second task is effectively observing the results of your actions. This does not cause us to start executing the actions again. But it does allow us to add changes in magnitude or speed to the ongoing action sequence or substitute an alternate action. If we just observe the sequence of stimuli caused by our actions then it is similar to being entertained. This self-entertainment should match the same sequence of stimuli that should occur when somebody else performs the actions.
This means that when a reflexive action is done to avoid boredom (obtain a change) and the action is done subconsciously the new sequence of stimuli (trigger and goal) form a novel / interesting conscious STM S-Habit. Then when the trigger occurs again we may perform the same action to obtain the sequence even if the interest in the goal is neutral. We try to reproduce the sequence of trigger and goal S-Habit because it represents the change and that is still novel / interesting.
Conscious STM
This means that the STMs that are currently forming sequential independent stimuli out of sequential dependent ones should keep their independent stimuli and chunk them to form the longest non-repeating sequence of independent stimuli. Longer novel / interesting chunks will then attract attention before shorter ones. This means that the change incorporated into the sequence is more interesting than no change. For example let us assume B and C have become independent. If the B, C sequence occurs for the first time when the C first occurs then they will both be novel. But the sequence will attract attention because it is longer. And it will only be after these longest sequences have been explored with actions that the independent parts should be explored. Read 3rd Aug 2010 for a similar idea.
9th Feb 2011 Expectations
The sequences of dependent stimuli that I form are really all the S-habits that have orienting responses between the stimuli. Each dependent stimulus or sub-sequence can have one or more next expected stimuli. They only become independent when a stimulus or sub-sequence repeats. This is when a non-orienting response should be done. This means orienting should not be one of the responses done to independent stimuli. The execution of the sequence in a dependent stimulus should neutralise the change that occurred. Thus only unexpected change should remain. This would be the sign that a novel sequence has occurred. Unexpected change from an orienting response will result in the novel sequence and no action will be performed. However as soon as the sequence repeats it becomes independent and responses are done to try and get a different next stimulus i.e. an unexpected one - a change that was not neutralized - a novel sequence that is interesting. Once a stimulus sequence becomes independent and as a trigger a response has been done and a familiar goal stimulus obtained a new sequence could have been formed. In this case the response is worth repeating because the goal sequence was interesting. What if a familiar sequence occurs when the response is done? The trigger & goal sequence is familiar and thus not interesting. The next time the trigger is perceived the same response will not be done. Will an orienting response be done waiting for a repeat of the trigger to cause the next reflexive response to be tried? Or will a reflexive response be tried because all known sequences are not interesting?
However we must have orienting habits that recognize repeating sequential stimuli so that the changes they produce can be neutralised. Are these habits only realized after we have explored all other responses using the stimulus as a trigger? We must also remember that an object even if it is spread over time is a pattern of changes that is repeated.
10th Feb 2011 Familiarity / Interest
When a stimulus occurs for the first time it is novel. Upon the second and subsequent occurrence it is familiar. The second time it occurs it must have one expected next stimulus. On subsequent occurrences it has at least one or more expected next stimuli. But a stimulus does not become interesting until it is first paid attention to. Before being conscious its interest level is unknown. Upon first becoming conscious it becomes interesting. Upon the second and subsequent times it is paid attention to it becomes and stays at neutral interest. However if it repeats it is boring. This is because no change occurred. This applies to the longest sequence of stimuli. The sequence CDE can occur many times as a familiar stimulus with many next expected stimuli and never have been conscious. It becomes interesting when it becomes conscious for the first time (consciously novel). On second and subsequent times when it becomes conscious it is of neutral interest (consciously familiar). A stimulus never becomes boring. However we are bored when we are conscious of a repeating stimulus. It is at this point that we perform a reflexive response in order to cause change. So how do we set up a subconscious habit to disregard a repeating stimulus without first trying all possible reflexive responses? This must happen first on all senses when repetition occurs. It's similar to combining adjacent sensors when they all have the same reading. This is also the basic principle of habituation. Thus a repeated sequence results in no change from the sensor for the longest current sequence because a looping orienting recognition habit is being done. Otherwise the longest sequence is either novel or familiar.
Novel longest sequences will attract attention first because they represent unexpected changes. Given any new or familiar sequence it is expecting itself to repeat and will reduce its change to neutral if it does repeat. If it repeats it will only attract attention if there is nothing else that changed or is interesting.
11th Feb 2011 Reflexive action
I believe the strategy from the bottom of the 1st paragraph from the 9th Feb 2011 is the correct one. After a trigger stimulus has repeated a reflexive response is produced. If the goal stimulus is interesting this response will be repeated the next time the trigger occurs and the goal stimulus should lose its interest level. Now when the trigger is perceived the next possible response will not be tried. The no-response (orienting response) will occur, just in case the trigger does not repeat. If it should repeat then the next possible response will be tried reflexively. It is as though the 1st time the trigger happens it is a result of a change in the environment. The next time it occurs, as a repeat it must be because attention was paid to the trigger's sense and the repeat was conscious. Conscious repetition is boring and causes a reflexive response.
I've also decided to make permanent stimuli, which are also reset to dependent, not boring so that no action is done at random when they occur. They are just combined with other dependent stimuli.
12 Feb 2011 Binon structure
On the 20th Dec 2010 I further clarified the binon structure. Now I have realized that a stimulus / an experience (What) is made up of a where piece and a value piece. The where piece consists of the tree with the top layer being the External value, the second layer being the tree of senses and the third layer being the tree of sensors. The value piece is just the lowest level readings combined based on the same where structure. One cannot compare value pieces to obtain a match. One must use the complete what piece because the where part only provides the identity. Then sequences of stimuli should be sequences of experiences. To display the combined readings one must decompose the value piece in synch as one decomposes the where piece. Experiences are the pieces that have interest and change flags. So when the change is neutralized in a higher level stimulus the parts must be reconstructed by dividing up the where and the value pieces and combining them at the part level. The change must then be neutralized on these parts.
Now I realize that each binon should have a where piece and two parts. Each part should be another binon with a where and two parts. Where pieces should consist of the External flag, Sense P-habit and sequence trees, sensor trees and other patterns (contrast, size). This way each part points to its two sub parts and each part has a where structure.
14th Feb 2011 Contrast & Shape independent
The contrast pattern of a combination of parts (an object) is independent of the shape (size) pattern. The contrast pattern could be a level 3 (consists of 3 intensities) while the shape pattern could be at level 6. That is the object could have 6 parts but each pair of adjacent parts could have the same contrast ratio (3:4) but the 3 pairs could match a level 3 contrast pattern.
Since we can pay attention to shape independent of contrast and vice versa, I need to devise a data structure that will support this. It must be similar to the independence between senses. It would mean that contrast and shape are combined in parallel rather than sequence. What about the values that are associated with a shape. These are the size and the rotation values. We can pay attention to the rotation of a shape or a shape with any (independent of) rotation. But the shape is always selected. Or is it? Can we pay attention to all part pairs where the part on the left is always smaller than the part on the right? Probably. We can also pay attention to size independent of rotation. This would seem to imply that the where shape information must be a parallel combination of the shape pattern, the rotation property and the size property.
21st Feb 2011 Parallel versus Sequential
Re. 3rd July 2010 and 4th March 2010, After discussion with Ann I have come to the conclusion that we form P-Habits first and then form sequences of these combinations rather than the other way around. Examples are that we can understand and recognize what is being said without seeing the speaker. We can understand and recognize what is being said and it looks normal when we see someone speaking to us but we can't lip-read because we don't create the visual sequence independent of the auditory one. We don't practice lip-reading and rarely have a need to. If we were forming the sequences and then combining them in parallel the visual lip-reading sequences would be natural and immediately recognizable.
24th Feb 2011 Interest in sequentially dependent stimuli
I now have P-Habits forming from single frame sense objects and now must determine what attracts attention. I have decided the longest sequence takes precedence over the largest P-Habit that takes precedence over the largest sensor combination of parts in any single sense. But both interest level and unexpected change attract attention. I have decided interest level in an object takes precedence over the largest combination / sequence that had an unexpected change. But this has led me to the conclusion that sequentially dependent stimuli do not have an interest level. They remain novel until they become independent (conscious as far as action habits are concerned). This is because interest level in a stimulus is only relevant as far as action is concerned, e.g. avoid or pursue.
28th Feb 2011 No Change
When all changes are neutralized by active habits and there are no interesting stimuli to attract attention my current strategy is to pay attention to the largest P-Habit stimulus or the last sense. However this fails to pay attention to the stimulus that had changed but was neutralized by the current habit. This was documented in the test case #20 run on 26th Feb 2011. I've decided to try a strategy where the largest changed stimulus before neutralisation attracts attention in this case. It is equivalent to paying attention to what the current habits expected.
5th Mar 2011 Attention attraction
From the test runs today, I have concluded that short sequences of larger P-Habit combinations have precedence in attracting attention over longer sequences of smaller P-Habit combinations.
7th Mar 2011 Interest propagation
On the 10th Dec 2010 (also 28th Aug 2006) I wrote about expected interest that a stimulus (A) has because it is associated with an interesting goal (B) via some action habit. The question becomes, when does a trigger stimulus (A) gain its expected interest? Is it (1) when the trigger stimulus (A) repeats and the expectation of the interesting goal (B) is thought about? Or is it (2) when a response (R) is done and an interesting goal (B) is experienced?
(1) The trigger stimulus (A) could gain its expected interest when it is experienced and the known action habit (A -> R -> B) is thought about with an interesting goal stimulus (B). When the action habit is done, B will have a neutral interest but A will have retained its expected interest. This does not seem too useful because it is no longer worthwhile trying to get to situation A. However, it would work for pleasurable emotions because B would continue to generate them every time it was experienced.
(2) On the other hand, a neutral trigger stimulus (A) could gain its expected interest when a response (R) was done and got an interesting goal stimulus (B). It would also lose this expected interest and become neutral again when the response (R) was repeated in order to obtain the goal (B). This is because the goal stimulus (B) would become neutral on its second experience. But in between these two the trigger (A) would act as a reason (interesting goal) to do any other action habit that might achieve this trigger. When this other action habit (trigger stimulus (C), response (T), goal (A)) achieves its goal (A) its trigger (C) might also gain expected interest. Then having reached the goal (A) the action habit (A -> R -> B) would be done. The experience of B would then neutralise B and A's interest level. However C would have expected interest which does no good. This would imply that expected interest in a goal should not propagate backwards to the trigger when the goal is reached. Maybe expected interest is propagated backwards to a trigger when thinking about the goal using approach (1). This could explain how more indirect stimuli such as a casino tokens gain expected reward.
The second (2) approach is also supported by the concept that when an action habit is rewarded all its parts (trigger (A) and goal (B)) are labelled with that reward. If a clicker were used to signal that the response was correct it would be experienced at the same time as the goal stimulus resulting from the response. A clicker can only have expected reward. Given the idea that expected reward only propagates backwards when thinking the clicker's expected reward might only be associated with the goal at time of occurrence and have to be thought about to be associated with the trigger.
9th Mar 2011 Mood and Motivation
A mood is a state that you are in for a period of time. Moods include bored, hungry, tired, thirsty etc. A motivation is a desire to change from your current mood to a future mood. Therefore a motivation could be the expectation or thought of a future desired state. When one is in a mood the emotional value associated with stimuli changes such that certain stimuli attract attention that would not otherwise. E.g. when one is thirsty and is motivated to escape from this mood, water will attract attention more readily. Is this because the motivation has caused the recall of the idea of water and thus "primed" ones attention to look for it?
A-habit execution structure
I need to know what the most recent acton was that was done in the execution of action habits so that should a habit fail this most recent action can be used to create a new action habit based on it and the most recent stimulus. An example in more detail is an A-Habit equal to St - 7 - Sg. Where St is the trigger stimulus, 7 the acton number and Sg the goal stimulus. The acton tree is:
Acton # Response 1 Feedback Response 2
1 a primitive
2 b primitive
3 c primitive
4 d primitive
5 1 S1 2
6 3 S2 4
7 5 S3 6
Where S1, S2 and S3 are feedback stimuli
When the A-habit is started the active sequence will contain: Sg St 7 6 S3 St 5 2 S1 1
This is the actons in reverse order going down the left side of the tree. St is inserted before the expansion of each acton so it can be assigned to the most recent stimulus that has occurred. The most recent response will be 1 and St the most recent stimulus. The 1 will be removed because it has been done.
When S1 occurs it will be removed and 2 will be examined. If it is a primitive then it will be done and removed and the acton before it (5) will become the most recent response. This will be removed as well. St. will be assigned as the most recent stimulus and removed. The new sequence looks like: Sg St 7 6 S3
When S3 occurs it will be removed and 6 will be examined. Since it is non-primitive it will be expanded and replaced by its parts (4 S2 3). If 3 were a non-primitive it would be further expanded. S3 will become the most recent stimulus and inserted before any further expansions. 3 will be done and removed. It will be the most recent response. The new sequence will look like: Sg St 7 4 S2
When S2 occurs it will removed and 4 will be examined. Since it is a primitive it will be done and removed. The acton before it (7) will become the most recent response. This will be removed as well. St will become the most recent stimulus and be removed. Now the sequence will look like: Sg When this occurs, the habit is complete.
When an unexpected stimulus occurs, the A-Habit will fail but the most recent response and most recent stimulus will be used to update the action habits. If the most recent response is a non-primitive acton then the action habit based on its feedback stimulus as trigger and 2nd response will be updated too.
12th Mar 2011 Orienting response
The ideas from the 9th and 11th Feb 2011 beg the question, should orienting responses to independent stimuli be kept as A-Habits? Or how do we form and recognize sequences of independent stimuli? When we have performed a reflexive response to a new independent stimulus we have an A-Habit (T-R-G). When the trigger occurs again the A-Habit is done subconsciously such that we become aware of the sequence of the two independent stimuli. This is when our conscious STM needs to start forming sequences of independent stimuli. These sequences should be monitored for repeats as well. Read 7th Feb 2011.
No stimulus
A possible idea is that I could have a stimulus that represents no input, e.g. silence. Read 21st May 2010. It would be kind of like the space between words. It would separate two noises. It could represent nothing being touched, smelt or tasted. It is rather similar to the concept of visual background for simultaneously seen objects. However I have realized that background is in the mind of the beholder since letters can be black on a white background or white on a black background. It is the identification of the objects in the scene that determines the remainder that is background. And the objects in the scene are familiar and recognized first while the background (object) is extremely complex, never the same as any other background thus unfamiliar and not recognized. Background ends up being the combinations of ignore gaps between the familiar objects. So do I need or should I use a sequential gap concept that is ignored as a no input stimulus?
13th Mar 2011 To Do - Solve
The following are things to do / solve:
- Add emotional values of pain and pleasure with moods.
- Work out thinking based on attention to actions and imagination.
- Add multiple output devices and have A-habits perform responses in parallel - 7th Feb 2011
- Work on graduated senses and motion detection.
- Get generalization / discrimination working as applied to problem solving.
- Get thinking to kick in when not interested in the results of A-Habits - 28th Feb 2011
- Need habituation of repeats in STM if subconscious - 10th Feb 2011
- Implement looping A-Habits - 25th Jan & 29th Jan 2011
- Add a conscious STM and memory for sequences of independent stimuli - 12th Mar 2011
Independent P-Habits
From the 13th Jan 2011 discussion the following is the algorithm for recognizing independent P-Habits. A stimulus is PI when it is parallel independent. That is independent in parallel with other stimuli. To make this work every stimulus needs a PI pointer. If it points to itself the stimulus is PI. Else it points to the PI P-Habit of which it is part. All the parts and subparts in the pointed to PI P-Habit are interdependent and form the unique combination of this PI P-Habit. It is always a lowest level PI P-Habit stimulus. When a P-Habit stimulus becomes PI any of its parts or sub-parts that match the lowest level PI P-Habit it used to be part of are changed to point to the new PI P-Habit. Any of its parts or sub-parts that don't match what it used to be part of are changed to newly PI.
First go tree wise through all parts of the largest P-Habit (L) found in the current frame and mark them not done. Then make L PI by calling the following recursive routine.
When a P-Habit (X) is made PI
Keep X's pointer value (V)
Set the P-Habit (X)'s pointer to itself (X)
If the P-Habit is new then - V will be empty so
Set the kept X's pointer to X - now V will = X
End if
Go through its not done parts from most complex to least, a layer at a time (not tree wise)
For each part (M)
If it is not done then
Mark this part (M) done
If the part (M) has no pointer then - it (M) is new so
Set the part (M)'s pointer to the newly PI P-Habit (X).
Else - M has a pointer (W) so
If W points to itself (M) then - it is already PI so
Mark all its (M)'s parts done
Exit this recursive routine
Else if W points to the same previous P-Habit (V) then
- M is a dependent part of this newly PI P-Habit (X) so
Set the P-Habit (M)'s pointer to the newly PI P-Habit (X)
Else - W points to some other PI P-Habit so make M PI
Call this routine on part M
End if
End if
End if
End loop
End recursive routine
Reacting to Sequential Independence
A stimulus is SI when it is sequentially independent. Reflexive action and performing action habits only applies to SI stimuli. The SI stimuli are the things we consciously explore because they are the biggest complex objects. We wait for the SI stimuli to finish before we start exploring them. Thus they become the triggers and goals. How do PI stimuli fit in? I think the changed parts of a frame first attract attention and are assigned PI as a whole because they changed independently. This is not based on the above top down PI algorithm. But when they are assigned PI the above algorithm kicks in for any sub-parts of this whole changed part. All the unchanged parts can then be assigned PI based on the above algorithm. If the changed parts were expected and interest neutralized then the next thing to attract attention is anything with interest.
Current attention
The current attention algorithm is based on the following priorities.
- Biggest stimulus with unexpected change
- Biggest stimulus that has an interest level or an expected interest level
- Biggest changed stimulus before habit matching
- Biggest SI stimulus
- Biggest dependent (not SI) stimulus
14th Mar 2011 Parallel Independence
When it comes to forming sequences of stimuli the rule should be that only PI stimuli are used. If a stimulus is not PI then it is not broken out of its configuration to form a sequence. Thus given the first frame from two sensors on which the pair P-Habit is PI but the two sensor stimuli are not PI the only stimulus available for attention should be the PI pair P-Habit. If the same holds true for the second frame then a sequence of the two PI pair P-Habits is formed. What if the stimuli from one sense become PI but not the other? Based on current inter-sense P-Habit sequences any sequence being formed is terminated and new sequences started based on the new senses that have changed.
An object that is PI can be composed of parts that are both PI and not PI. The subparts only occur in the particular configuration for this object. But a lowest level PI object has parts that are only non-PI and only occur within it. That is to say a parallel dependent object can only be part of one lowest level PI P-Habit. Is it possible to change the algorithm for recognizing PI stimuli to a bottom up approach rather than top down? The first logical thing to do would be to make new stimuli non-PI by default since they only appear in the P-Habit that will be created. This would empty their PI pointer. The next thing to do would be when a pair of stimuli (X and Y) is being combined to form Z check their PI status and pointer. If both X and Y's pointers are empty then change them to point at this new Z being created since they are only found in this new pair Z. But upward pointers to the next level will not work because a part is paired with many parts on other sensors at the same time. A pointer to even higher levels was required in the top down approach to capture the full set of other sensor involved. What about using two pointers in every Z to its two parts? If there is a pointer to X then that part X is only used in this Z and if no pointer to X then X is PI and used in more than one pair based on the same two sensors? The problem is to know when a different pair reuses the same pointer to X as an existing pair.
Let us consider these concepts for a sense that has dependent sensors. Vision will be easier to think about. If we are a baby with no past experience and presented with a novel scene all the parts in the scene are dependent in parallel. There is no combination of parts that is PI except for the whole scene. If something in the scene then moves, this subset of the parts is recognized as a PI object. It changed independently of the rest (background). But also the remaining background that did not change is one big PI object. What about the new parts, the ones taking up the space where the moving part came from? For example the first scene is 231854MNW725685432 and the MNW moves to a position replacing the 568. The new scene is 231854ABC72MNW5432. The ABC is new. The 568 has gone and the MNW has moved. MNW is familiar and given that it has moved would become PI. The background 231854---72---5432 would be PI since it did not change. ABC would be a parallel dependent part of the whole background 231854ABC72---5432 that is PI. If the PI MNW appears in any other scene it would never be absorbed as part of the background because it is familiar as an object worth responding to. And as long as MNW continues to move in a non-repeating sequence we would not explore it. As soon as its pattern of motion is predictable then reactions would occur.
But now what if a second part of the scene starts moving at the same time? If the two parts maintain a separation relationship to each other they will be seen as two parts of the same object and therefore dependent in parallel. The same would happen if the changes were not of location but of intensity or size. But if the motions are not in synchronization because the separation is changing then the two parts are seen as PI.
15th Mar 2011 Sequential Independence
The idea of a conscious STM is to create sequences of SI stimuli. The concept is to keep track of "what happens next". Given any SI stimulus there could be many what happens next stimuli. Mind you this also applies to non-SI stimuli. So what is the real essence of the distinction? Currently a stimulus becomes SI when it repeats or ends with an existing SI stimulus. The repetition is a sign that the sequence is no longer entertaining. It is no longer unpredictable. Unpredictability implies there is novelty (unfamiliarity) in the sequence. Since the drive is to avoid boredom and pursue novelty there is interest in unfamiliar sequences. But maybe the definition of SI should be consistent with that for PI. A stimulus that is non-PI is part of only one PI stimulus. It is dependent on at least one other parallel stimulus. It always occurs with at least one other stimulus. This means it is not singled out as independent for recognizing its change in a sequence. The combination of it and the other one or more other parallel stimuli make up a PI stimulus for change recognition. And since PI stimuli can be combined with other PI stimuli to form more complex combinations these combinations may be non-PI. For example consider the combination ABX in which AB could be dependent in parallel on the X. A, B and X could all be PI because they occur in different combinations such as ADX, CBX, CDX, ADY and CBY.
Applying the same idea to sequential dependence would be that a stimulus is non-SI if the next stimulus is predictable - has only one possible value. But this does not apply. The principle is to recognize the longest sequence that repeats as a unit in which there are no sub-sequences that have repeated twice in a row. As soon as they have repeated twice in a row they become SI and parts of a sequence that can be explored separately from a larger sequence in which they may be found. Thus if AB becomes an SI sub-sequence after a previous sequence such as SEFABCFEABD was found to be SI the next time this long sequence occurs it will not be recognized as a whole. It will consist of the sequences SEFAB, CFEAB and D(?).
Further thought has produced the idea that reflexive responses should only be produced when a repeat is detected and no action habit exists with an interesting goal. Thus when AB has become SI and we have already tried the 1st reflexive response with no interest in doing it again and sometime later we get another AB we should not try the 2nd reflexive response. We should only do this if the AB repeats and no interesting action habit is known. This is sort of why I introduced the idea of an orient response between each try of the next reflexive response. Only producing a reflexive response when the AB repeats and there is no desirable action is then consistent with the idea that a reflexive response is a reaction only when boredom occurs. We still have to retain the idea of the longest non-repeating sequence as the unit of conscious responding so that only after the third AB in the sequence ABABAB do we perform a reflexive response. It is only after the end of the second AB that we have an expectation of what follows AB and can predict this. It is when the prediction of what one can expect next is realized that we are bored.
But we need to still deal with predictability of "what happens next" when the sequence is being composed of SI stimuli. If AB, CD and EF were SI sequences of stimuli then we would continue to be entertained even by any combination of these even if they are all known to follow each other. This is easier to think about if we assign M = AB, N = CD and P = EF. M expects both N and P next. N expects both M and P next. P expects M and N next. So there are no combinations of two of M, N and P that are not known. But given the sequence MNPNMNPMPN we continue to be entertained because there is no repetition. The length 3, 4, 5 etc sequences are being followed by sequences we do not expect and thus novelty is occurring at higher levels of complexity (lengths). Note that even though the sub-sequences such as PN or MP are familiar and are SI no responses are made. This is true even if action habits existed with these as triggers had interesting goals. That is because the immediate interest generated by the non-repeating / entertaining sequence is not superseded by the interest in action habits. If we were dealing with other motivation such as hunger a sub-sequence such as PN might trigger an eating action habit. So this rule only applies to the feeling of interest.
20th March 2011 Dependent stimuli
A couple of ideas I have had recently. Maybe parallel dependence between stimuli from independent senses or sensors is not important. Maybe it just comes out in the wash so to speak as a result of always combining together those stimuli that have changed from one frame to the next. If two stimuli change / occur at the same time and the combination is unfamiliar it will be remembered. If it occurs again it would be familiar. If we are always trying to recognize the biggest combination that changes then the familiar sub-combinations will be recognized first and combined to form the biggest familiar combination. Then the un-familiar combinations will start to be formed from these. Smallest familiar sub-combinations will retain their interdependence until their parts occur independently. Thus new unfamiliar combinations will be detected. The key objective is to pay attention to the unfamiliar combinations to better understand how the environment behaves.
Parallel independence between stimuli implies that the “what happens at the same time” has more than one possibility. All the known possibilities have to be learnt and disregarded so that the novel ones stand out and attract attention. And this has to be done at all levels of complexity. Likewise all the “what happens next” possibilities have to be learnt so that the novel ones (unexpected) stand out and attract attention. The classification into dependent and independent is too artificial. It’s just a case of remembering the combinations, recognizing / identifying them when they repeat and noticing when they are unfamiliar.
Parallel Actions
Parallel actions occur between devices. Each device can be given a command simultaneously. If each device is a muscle we appear to give them commands that are a value on a scale of how much tension to produce at any one time. The same is true in Adaptron. Each response includes an implicit amount of distance to travel. But they are symbolic responses rather than graduated. They also include direction information. It would be more appropriate to use two devices such as two wheels. One aligned north - south and the other east - west. Each wheel could be given 3 responses corresponding to forward, backwards and stop. These could be +1, -1 and 0 respectively. Then a more sophisticated wheel might be given larger values to travel further per unit of time similar to a greater tension on a muscle.
To represent parallel responses I will have to modify my R-S-R acton structure. Currently each acton has 3 pointers. The 1st and 3rd pointers point to other actons and the 2nd points to the intermediate feedback stimulus. There are primitive actons that are single responses for a single device. I need parallel actons that have actons from different devices as the 1st and 3rd pointer and an empty 2nd stimulus pointer.
21st March 2011 Subconscious A-habits
Since A-habits are started and run subconsciously they should not be used to decrease the unexpectedness of changes in stimuli. This should be all part of recognizing sequential and parallel patterns of stimuli. S-habits and P-habits should be doing this. They should be monitoring the results of action habits and making new responses that may modify the ones being performed.
23rd Mar 2011 Sequential Independence
But I still need some kind of distinction between the sequences that are consciously reacted to. I could say a stimulus is novel until it repeats at which point it becomes conscious and interesting. Novel stimuli would not be attended to. They would just be recognized as parts of sequences. When they are interesting and they repeat then they are boring and a reaction response is done. The goal stimulus that follows this response would have to be a conscious / interesting (previously called SI) one. This goal would be the longest sequence of novel stimuli until an interesting one occurs. What if the sequence ABC repeated? At the end of the 2nd repeat it would become interesting. At the end of the 3rd repeat the reaction would occur. Now if the next stimuli were DEF and this was novel one would have to wait for the sequence to terminate with a conscious stimulus. For example another DEF or EF or F. Once we have these conscious / interesting sequences they are going to become neutral on second occurrence (assuming they don't repeat immediately). Now that they are neutral and conscious they will be thought about and partial trigger matched to determine if there are any responses worth doing to obtain an interesting goal. If nothing can be found they must stay in STM and be combined with what comes next. Even if a response is found it will be started subconsciously and they must stay in STM and be combined with what comes next.
24th Mar 2011 Becoming SI
On the 2nd May 2010 I asked the question about a conscious sequence terminating a novel one. My 1st test run of today addresses the question. I have decided to try a strategy where given that AB is a conscious sequence and CDC is not, when CDCAB occurs CDC becomes conscious and the two stimuli CDC and AB are recognized.
25th Mar 2011 Subconscious sequence
On the 15th Mar 2011 I said the subconscious sequence SEFABCFEABD would be experienced as different after AB became conscious. The new interpretation would be SEF AB CFE AB D because the sequences between conscious sequences become conscious. The AB acts as a conscious divider or edge.
26th Mar 2011 Conscious Sequences
If reflexive reactions only occur when a conscious stimulus repeats, then conscious sequences must grow very long in STM. However should a conscious stimulus in such a sequence be recognized that has an interesting action habit then this habit would be started. When the habit is begun would the STM be flushed? When action habits are done they are subconscious and we consciously pay attention to the resulting stimuli or our mind wanders because it is thinking. When we pay attention to the resulting stimuli we are forming sequences of conscious stimuli. Any unexpected stimulus attracts attention and we may perform an action habit or modify the current one being done.
If action habits are not to reduce the interest or expectedness of conscious stimuli then the sequences formed by trigger and goal stimuli must become familiar when they happen and are conscious. Thus when a reflexive reaction is done or at least when an action is begun subconsciously STM should not be flushed. The new trigger - goal sequence must be created and attended to so it can be expected should the trigger reoccur.
8th April 2011 Conscious actions
I had a problem with STM creating longer and longer sequences of conscious stimuli that span actions being done by action habits. The action habits are being done subconsciously and were expecting the tree of stimuli making up the sequential goal stimulus. For example a goal stimulus sequence of ABC was expected as A then AB then ABC. However Conscious STM was building sequences that included the trigger stimulus before the goal and this did not match the action habit. So I have reduced the expected list of stimuli in the goal part of the active action habit to contain only the S-Level = 1 stimuli in the sequence.
But I have a problem with recording a new action habit when an unexpected goal occurs. I keep the previous trigger and action performed for this purpose. But the conscious stimulus in STM includes the trigger and this cannot be used as the unexpected goal stimulus. An action habit has a conscious trigger and goal stimulus. There is no recognition of the trigger and stimulus as a single sequence. But this is what STM is doing. For reflexive actions triggered by a repeated stimulus in STM the STM is flushed so there is a clean break between trigger and goal. Action habits are built based on this clean break. If these action habits are repeated, then this clean break must be respected for recording unexpected goal stimuli. It is as though the conscious start of doing an action habit causes this break and an expected goal is being waited for. However the sequence of stimuli from the occurrence of feedback stimuli, experienced inside the acton tree as it is performed subconsciously, are recognized in STM as a sequence. This idea also corresponds to the idea that we always have a goal in mind while awake.
This leads to the idea that we always have a conscious action habit in mind. Therefore when we are doing orienting responses / nothing action habits, does this same approach apply? I think it is what is happening in STM. It speaks up with an unexpected change for any unfamiliar sequence. But back to the idea that when we consciously start an action habit we set up an expected goal we are waiting for. This is a thought. Also at this time we should flush STM for the goal sense so that the trigger and goal do not form a sequence of conscious stimuli.
When we are performing a long sequence of responses in an acton we can be consciously monitoring the stimulus sequence (self-entertainment). Whenever we notice a deviation from what we expect in this stimulus stream we can start other action habits to subtly modify the action sequence. We can also start up in parallel other action habits. Provided they do not conflict with the resources being used by the active action habit we end up forming more complicated action habits that we can reuse as a single acton. But when we start up these in-parallel action habits we cannot afford to flush STM. Or can we flush it? It would seem that STM continues to combine stimuli independently of whether or not we are paying attention to its creations. Whenever we start thinking it is as though STM throws away any sequences of conscious stimuli but keeps subconscious sequences. Thus it only has the last conscious sequence when we come back to pay attention to it.
Unfamiliar sequence
Right now for sequences I use the unexpected change to attract attention. An unexpected change is one where a repeat of a previous adjacent sequence has not occurred. This is kind of like an extension to the concept of a change happening on a sensor between S-Level = 1 stimuli. However, it is probably unfamiliar sequences that attract attention. They are ones where the sequence produced is new. Familiar sequences do not contain any unexpected changes. It is this more general definition of unexpected change that I want STM to identify.
Change versus Unfamiliar
There is change detected by the sensors in Par() but their memory is limited to just the previous stimulus. There is change detected in STM by repeats of sequences but is limited to memory of what is in STM. There is change detected at the action habit level that is based on LTM when unexpected goals occur. Sensors can't do anything about change that they detect other than pass it on. Although with dependent sensors change is used to identify unfamiliar objects and motion. STM detects the same changes as the sensors but as a familiar or unfamiliar sequence. But if STM is flushed between conscious stimuli then a change or unfamiliar goal must be detected. While the acton is being performed subconsciously it must be the STM that is looking for unfamiliar sequences of conscious stimuli. But does STM build up higher S-level sequences of these stimuli or does it just validate that the sub-goal stimuli are familiar ones given the preceding trigger stimuli?
STM recognizes unfamiliar stimuli
So as we are performing the acton subconsciously STM is monitoring all stimuli for any unfamiliar sequences which are unexpected changes. The STM does not worry if the stimulus is the appropriate goal stimulus for the acton being performed. This allows us to put away the cereal in the fridge and the milk in the cupboard if we are not paying attention to what we are doing. Since thinking can divert our attention from STM many stimuli can go by without any awareness of them. STM though is still monitoring valid sequences so that it can interrupt thinking with unexpected stimuli that result in unfamiliar sequences.
Performing action habits
On 20th Jan 2011 I asked - How do we stop stimuli in the sequence from initiating the lowest level action habit that is already in progress? The stimuli are coming in from the acton execution. The STM is recognizing them and determining they are familiar. A possible example is I am on my way to the kitchen to get a vitamin C when I see the broom leaning against the door. I decide it needs to be put in the closet. I grab it and put it away and then go on to the kitchen. This is not quite the same thing. This is an example of where an unexpected stimulus attracts attention and we start a different action habit. It interrupts the current one. I want an example where we are conscious of the stimuli being produced by the acton execution and have not decided to fine tune / change the responses. Because we have a goal in mind does any desire to initiate a response need a higher level of interest / reward than the current goal? Or is it that none of the intermediate stimuli have an association with the goal. They don't result in thoughts about the next stimulus and its desirability. Or that the desirability of the next stimulus is neutral. We are down at a lower S-level of stimuli.
An example might help. Trigger stimulus T with action b results in goal G. G is desirable. The LTM action habit is T b G. T happens again. We think of G and T gains expected reward. We repeat action b and get G. Later trigger stimulus S with action f results in T. The LTM action habit is S f T. When S happens again we think of T and S gains expected reward. Thinking about T reminds us of G and T's expected reward is confirmed. We now start doing f and get T. We then do b and get G. The acton f T b (#28 for example) is produced. LTM of action habits should now contain S 28 G. This is not giving me what I want. Maybe the solution is that all the feedback stimuli obtained are kinesthetic. And kinesthetic stimuli are not associated with stimuli from other senses that produce desires. All kinesthetic stimuli and sequences of them must therefore be known, otherwise interest is produced. Even so this does not solve the original problem. The feedback stimuli for the acton could all be kinesthetic but it may be an expected and experienced visual stimulus during the acton execution that prompts us to start doing the remainder of the action habit already in progress.
9th April 2011 Thinking ahead
During the thinking that takes place that identifies a desired goal the idea of the desired goal is given an expected interest that is "felt" when that goal is reached.
Long term goals
When the goal is a long term one, such as go down stairs to get the mail, there are many intermediate trigger stimuli that start subconscious actons. Opening a closed door is an example. If the door were already open one would just walk through and not do the open door acton. When the open door acton is started the interest in the goal is neutral and no feeling of success is obtained when it completes. It is done completely subconsciously. However the get the mail acton is the one with the interesting goal. We are conscious of this action habit. The open the door acton appears to be a recognizable alternate sequence in getting the mail. However it is independent of getting the mail and only done if the door is closed. Even the door closed trigger stimulus can be subconscious and it still gets done when getting the mail. Thus it would appear that many possible stimuli sequences could happen in the process of getting the mail. This means that given trigger T and response r there may be more than one goal stimulus that could occur. Based on the goal stimulus that does occur and is experienced thinking must be done about what response to perform to continue. In the open the door habit the response is to grab the handle and push. However this may result is the door not moving. This produces the idea that it is locked and we start the get keys from pocket habit to unlock the door.
Sub goals
It is as though when one thinks about reaching a goal that requires two or more conscious steps one sets up a list of the goal and sub-goals to be reached. If there are just two steps (responses) then the list consists of the sub-goal and the goal. If the 1st response allows the sub-goal to be reached then one performs the 2nd response to reach the goal. If the 1st response does not achieve the sub-goal it does not cause the overall habit to fail. Instead we have a different starting situation (trigger) from which to achieve the sub-goal or reach the goal if it can be reached in one response. This means the desired goal remains. So when do we decide the goal is not reachable? We could have a different goal replace it due to thinking.
So when the 1st response fails to reach the sub-goal we now have a current situation, a sub-goal and goal. We are now asking memory for any action habit that will get to the sub-goal from the current situation. If the sub-goal has an expectation of reward then this will be a standard recall of the current situation and finding the action habit that will achieve this sub-goal. If an action habit exists with a goal that is of higher interest, (e.g. interest rather than expected interest) then this may be done instead.
So, can this be accomplished by giving all the sub-goals involved in the habit execution levels of interest which they lose once they have been reached? This would mean the thinking ahead process assigns the expected interest in the sub-goal of a two-stage action sequence. And the habit execution reduces this expected interest to neutral. Or is the expected interest reduced to neutral if a feedback (sub-goal) stimulus is attended to. Does this address the question of the intermediate feedback stimulus caused by the subconscious execution of a learnt acton causing the remainder of the current acton to be restarted? I think so, provided the acton execution does not do it. What about a sub-goal stimulus that had expected interest but was not attended to during the acton execution. Does it retain the expected interest?
Let us consider the two stage acton sequence formed from the two habits S f T and T b G. And let us assume the f and b are actons rather than primitive responses. The goal G is fresh or interesting. Trigger stimulus S is attended to. Thinking about T results in it getting expected interest and the f acton is started. If we reach T then it becomes neutral and the T b G action habit will be started. If we fail to get T and get M instead T will remain expected interesting. We may or may not have action habit experiences to get from M to T or to G. If we do we will try them. But what about the feedback stimuli within f? Any of them that are attended to that have action habits with related goals that are interesting should cause us to start these other habits. Should they also interrupt (or run in parallel with) the current habit if their goals are expected interest? And how much thinking ahead should take place on these attended to stimuli? Have their goal stimuli got no expected interest or interest? Maybe their goals are all neutral because they have been fully explored.
20th April 2011 Actons
An acton appears to be a group of some kind of all the actons that are acceptable behaviour given the acton initiated by a conscious trigger. All the acceptable stimuli that could be encountered and the acceptable order of them are known and the responses to be made are practiced and known. It is only when the feedback stimuli deviate from the acceptable that conscious processing is necessary. The group appears to be a tree where the actons may be in parallel or sequence. I think the RSR structure might be a little too constraining for this purpose. The units need to be smaller and more dynamic. Would an RS combination do the job? Or would an SR combination do the job so that the stimulus invokes the appropriate response?
Actons are stripped of their goal stimuli because they have no need to make decisions about which action is better. They make the decision based on a match to the trigger. However they still need to be grouped / structured appropriately in sequence and parallel. And they need a hierarchical decomposition down to the lowest level. If they are made up of an S and an R then the S can be a small subset of the conscious trigger that was used to initiate them. They must at least have an R piece and this must be the acton identifier for the next level down. They must have a stimulus piece that is used as a match to perform the response. Could the response piece be an orient / no response - possibly if the stimulus is a sequential one? The tree structure would mean that an acton is comprised of two actons that are either done in parallel or in sequence. But also it has to have several choices for possible stimuli at any point in a sequence depending on what happens at that point.
21st April 2011 Acton structure
I will use A to represent an acton, R to represent a primitive response and S to represent a trigger stimulus. The lowest level acton A1 must be an S1, R1 pair. Then at this level it is possible that there are numerous possibilities that could be initiated in parallel. Thus the pairs A2 = S2, R2 and A3 = S3, R3 could also be performed provided that S2 and S3 are also found in the current trigger situation. Thus at the lowest level the group A1, A2, and A3 provide a set of possible things that could be done if their stimuli are matched. There is no sequence involved here. And there is no mutually exclusive concept in which only one can be picked. The stimuli do not need to be on different senses, although they could be. The stimuli could even be opposites such as S1 = black and S2 = white and then only one of R1 or R2 would be done. This structure provides the concept of a group of possible actions and the ability to allow for actions being done in parallel.
So now the combination of A1, A2 and A3 must be represented and uniquely identified. It may grow as alternate and / or additional actons are added to it.
We could combine A4 = (A1, A2), A5 = (A2, A3) and A6 = (A4, A5) based on binon formation rules. If the trigger stimulus is the symbolic value X on a single sensor sense and primitive responses are u for up, r for right, d for down and l for left then the following could occur. S1 = S2 = X. A1 = Xu and A2 = Xr. Then A4 provides for two parallel actions that can be done when receiving stimulus X. X and A4 would only be associated though once the two responses had been tried in parallel.
We also need to form sequences of these actons. So let us assume actons A11 through A16 are parallel / choices for a single time frame. A17 would be a sequential acton if A17 = (A11, A12) provided that once A11 was done we would wait for the S12 of A12 to occur and then do R12. The R11, S12, and A12 are very similar to my previous RSR structure but with the initial S11 included.
22nd April 2011 Acton structure
I could use a binon type (Acp) for acton parallel composed of two acton primitives (Act). Then two parallel actons (Acps) could be combined to form an acton sequence (Acs). Thus,
Acs = Acs & Acs
Acs = Acp &Acp e.g. A17 = (A11, A12)
Acp = Acp & Acp e.g. A6 = (A4, A5)
Acp = Act & Act e.g. A4 = (A1, A2)
Act = S & R e.g. A1 = (S1, R1)
But does a tree structure make it harder to add or remove an acton. The cerebellum appears to use long lists of Acts to form the Acp and turn the Acts on and off with climbing dendrites. But the sequential actons would still form a tree structure. So this is what I will try.
Acs = Acs & Acs
Acs = Acp &Acp e.g. A17 = (A11, A12)
Acp = Act & Act & Act & Act etc. e.g. A6 = A1, A2, A3
Act = S & R e.g. A1 = (S1, R1)
23rd April 2011 Acton structure
In the 'go to kitchen' acton there are many 'whenever' and looping actons. Examples of the whenever actons include the 'open door' acton that is waiting at any point in the process should a door be closed. Or there is the 'turn light switch on' should a dark room be encountered. There is the 'step over' randomly placed toy should one be encountered. An example of the looping acton is the looping 'walking' acton that is begun and restarted anytime the right conditions hold. This implies the acton at the highest level is a list of many actons all ready to be started or restarted given the right trigger conditions. The 'go to kitchen' acton should be able to be started from the initial condition of 'sitting in office chair', 'standing in bathroom' or 'lying in bed'. As a result of practice one has learnt that the 'proceed to kitchen' acton must be preceded by the 'get out of chair' acton, 'exit bathroom' acton or 'get out of bed' acton to form a subconsciously executable 'go to kitchen' acton. This would seem to imply the 'go to kitchen' acton is a parallel set of complex actons that are available to be done at any time during the action sequence. The order in which these complex actons get done is determined by the trigger stimuli encountered. They are all 'interrupt driven' based on the stimuli that are detected. The complex actons stay ready to be done at any time until the whole acton is complete. Completeness happens when we become conscious of the goal stimulus. It can't be when all the complex actons are not active because a looping one such as walk will continue until its resource requirements are constrained (trigger stimuli required to continue are not perceived).
Parallel and / or Sequential Actons
This all means that an acton can be a parallel list of actons and / or a sequence of actons. Thus at any level in its complex tree an acton could be a list or a sequence. The most primitive acton is a stimulus and response. Is it good enough that only the lowest level primitive actons have a stimulus. Should all actons at all levels have a stimulus? I think they all need a trigger stimulus.
Parallel versus sequential stimuli
This leads to the possibility that stimuli are recognized not only as sequences of parallel stimuli but also as parallel combinations of sequences of stimuli. If we consider the three sensors of the neck angle, thumb angle and ankle angle they are all independent of each other. They are each producing a stream of stimuli. If they are all operating independently then we have 3 parallel sequences of stimuli. If they are operating dependently i.e. they are in synchronization we have one sequence of 3 parallel stimuli. If the stimulus values are symbolic how do we detect this synchronization or independence? And which do we assume first if it cannot be detected? I have got a logical approach to detecting independent / conscious stimuli using a STM. It's based on repetition. Would two or more sensors be in synch if their conscious sequences started and terminated at the same point in time?
We need to start with parallel sequences because we start by assuming the sensors are independent. Then when the synchronization starts showing up we can start forming sequences of parallel stimuli. This would mean that when we start to recognize the start of a known to be conscious stimulus on two or more independent senses/sensors we begin to form the P-habit sequence. Should any two of these finish at the same time then we form the P-Habit (parallel combination of sequences) as a conscious stimulus. But since we do not learn to lip read by simultaneously watching and listening people speak it would imply that the individual sequences in these parallel combinations are subconscious. This is the same as the simultaneous parts of a whole that always occur together within dependent sensors.
27th April 2011 Forming sequences of Actons
As noted on 8th of April, 2011 we have a conscious habit and goal in mind at all times. This also applies to sequences of action habits. It is a conscious process that is used to combine two action habits into a new single acton. We cannot expect the subconscious performance of action habits to do this because they do not include any concept of goal stimulus. And the feeling of success and failure, when a goal is reached or not, is necessary for forming action habits. The conscious algorithm must save a successfully performed action habit as the previous one. Then when this is followed immediately by another successfully performed action habit the two must be combined into a single new action habit. The event of 'followed immediately' is detected when the goal of the previous one is the trigger for the second.
STM should be combining conscious stimuli into recognizable sequences while action habits are being performed subconsciously. This is necessary to detect unfamiliar stimuli and make them conscious so the unexpected stimuli can be used to form new action habits. But should subconsciously executing action habits be using STM sequences of conscious stimuli? It is possible that a trigger stimulus for an acton is a sequence of conscious stimuli. The acton has to wait for the sequence to happen before it performs its response. I believe I solved this on 8th April by having the executing A-habit expand the sequence into a list of S-level = 1 stimuli which it has to process. Thus the subconsciously performed A-habit does the sequential recognition. I don't believe this is what our brain does. I don't think the cerebellum recognizes sequences of stimuli. I believe it uses the power of the cerebral cortex to do this.
Now the challenge is how to represent a sequence of conscious stimuli. The difficult question is how to represent them such that when a conscious stimulus that is comprised of two or more subconscious stimuli is recognized as consisting of two conscious stimuli. What structure should be used for conscious stimuli such that they can be chunked into combinations and divided up at the lowest level.
28th April 2011 Conscious Stimuli
The solution is to continue creating all stimuli from binons of the lowest level stimuli. Then just change them from sub-conscious to conscious as necessary. Then a conscious stimulus that consists of numerous sub-conscious stimuli can be divided into two parts that are conscious by setting the state of these two parts to conscious. For example consider ABCD to be conscious but the individual letters, twosomes and threesomes are all sub-conscious. CD becomes conscious. This would cause AB to become conscious. However ABC, BCD, A, B, C and D would remain sub-conscious.
However on further thought about apples with pips and oranges with pips, when pips become conscious independent parts the parts of the orange or apple minus the pips does not become conscious. Thus in the ABCD example when CD becomes conscious AB remains subconscious. It must repeat independently to become conscious. This means that when the sequence ABCBC occurs BC becomes conscious but A remains subconscious. What happens to the ABC? If the BC should then occur again between sub-conscious stimuli does it make the sequence that ends with it conscious? For example consider BC is conscious and we experience MNBC where M, N and MN are sub-conscious. I think the sequence remains sub-conscious. Consider that X is conscious but all other letters are subconscious. The sequence AXBXCXDXEXFX would continue to entertain without any reaction from the Xs. If X and Y have become conscious then the same would be true for AXBYCXDY. However the sequence ABXY would result in the conscious Y being recognized and the ABX being designated as conscious. No.
I need to think of it at the basic principles level. A novel sequence means there are unfamiliar changes, unexpected ones that continue to be entertaining. A stimulus that has become conscious has been known to repeat at least once. But as long as it continues to be followed by unexpected stimuli, even conscious ones, there is no need to perform actions after it. It is unfamiliar sequences that continue to attract attention / entertain because unfamiliar means changes that are novel. So after a stimulus has become conscious due to a repeat (an undesirable familiarity) it is the next repeat that causes a reflexive action to be done. But what causes us to repeat this action habit. It should be when a familiar sequence reoccurs. Then the next expected stimulus for the trigger based on an orient response is not interesting. Can such orient responses be put in the NxtLnk list?
What is currently being done is that the goal from a reflexive response is neutralized if it is known. When the trigger re-occurs, the action habit is not repeated. Then when the goal stimulus is of neutral interest and it has occurred again (not necessarily in a repeat) it becomes boring. If the trigger occurs again the potential for a boring goal causes it to try something new. It only repeats the action habit if the goal has become interesting for some reason.
29th April 2011 Goal Recognition
The challenge is in the recognition of the goal when a reflexive action is performed. The trigger is easy to recognize because it has repeated and is conscious. It would seem reasonable that the goal stimulus would be the first unfamiliar stimulus that occurs after the response. This would be the entertainment achieved from the action. However what if the entertainment went on for many sub-conscious stimuli? Are they all to be designated as the goal?
Once an action habit is repeated the expected goal is known and can be matched if it reoccurs. It is also possible to recognize when something other than the goal happens. But again the question is how to recognize what comprises the new unexpected goal. From both the reflexive action and the unexpected goal situation we are trying to learn about how the environment behaves as the result of a response. What we have fundamentally learnt is the new relationship between the trigger and the next sub-conscious stimulus. Should the goal stimulus then be just the next atomic stimulus whether it is conscious or not?
30th April 2011 Goal State
In the literature it is often said that the goal of an action is to end up in a particular state. From S/W modelling a state is represented in the values given to one or more state variables. How does this translate into symbolic representation of stimuli? It would seem to be equivalent to a certain reading has to be obtained from a particular sense and sensor. If the reading were symbolic, this approach means that a certain symbolic value needs to be obtained. And this is why I have been calling it a goal stimulus.
Expected Goal
When performing a reflexive response we have no expected goal stimulus. But when repeating a past experience we have an expected goal stimulus. We can recognize it when it happens. But we cannot recognize the unexpected stimulus without some attention / where information. And the attention / where information is the senses/sensors etc. and the length of the sequence. The where-information should be the where-information for the expected goal stimulus. But on a reflexive response there is no ‘where-information’. Thus one would use the 'whatever attracts attention next' stimulus. Presumably this would be the change caused by the response. Ah. But on a reflexive response one is trying to avoid a potentially boring stimulus. So we do have an expected stimulus we are trying to avoid and its information about where can be used.
This means that for a reflexive response, triggered by the conscious stimulus AB, which is trying to avoid a repeat, the goal to avoid is AB. If the next stimulus starts with an 'A' we will wait for the second part. If it starts with anything else that single stimulus will be the goal. Using the same logic if the past experience is being done to achieve an expected goal then the expected goal sequence is matched up until the end for success and stops at any point in the sequence when there is a mismatch. At the mismatch point the habit being done has failed or in the case of the reflexive response it has succeeded. So what now is to be considered the new goal? Should it be the part that has matched so far plus the one extra stimulus that mismatched? Or could it be a sequence as long as the expected but no longer? But it could be shorter if a repeating section terminates it. This sounds reasonable. So given the example where one is expecting the goal to be ABCDEFG and one gets ABCDXYZMNOP the actual goal would be the same length as the expected and therefore would be ABCDXYZ. If one gets ABXYXYXYXYXYXY then the repeating XY would terminate the goal prematurely to the sequence ABXY. The same logic would be used in the case of the goal to be avoided to determine the successful goal.
This approach means that success of a reflexive response and failure of a repeated habit may be detected before the goal stimulus has been determined. But failure of a reflexive response and success of a repeated habit must wait for the end of the unwanted goal or expected goal to occur.
Interest Taxis
The current approach is a form of interest taxis. When something uninteresting is experienced or expected, a response is performed to hopefully produce an unfamiliar experience that will be interesting. When something interesting is experienced then no action is performed. When an interesting or expected interesting experience can be achieved the appropriate action habit is performed.
1st May 2011 Goal stimulus
Another possibility is that before the expected length of the sequential goal stimulus has occurred a conscious stimulus is encountered in the sequence and that becomes the goal. Thus we may not have to wait until the full length of the goal.
Reaction repetition
Right now when a reaction is done in response to a repeat the STM is emptied so that if the goal is another repeat the next action is to do nothing rather than react again. It is though it is waiting for the goal to be recorded before it sees if there is another repeat. Read 17th Dec, 18th Dec 2011. This seems correct. The reflexive reaction causes the trigger to stand-alone and to not be linked as a sequence with the goal achieved.
When a learnt action habit is repeated because the goal is interesting the action is done subconsciously. The goal will become familiar and neutral. But the STM should not be emptied as it was for a reflexive action and then sequence recognition will produce an interesting sequence from the trigger and goal. However the action done must somehow be part of the sequence so that we can think about the "what happens next, when".
2nd May 2011 Self-Entertainment
Self-entertainment is achieved when one observes the stimuli generated by a subconsciously performing action habit. Action habits are created when we react reflexively to repetition. The basic principle is to do the action reflexively to avoid an expected repeat or react to an actual repeat in order to cause change and hopefully something unfamiliar which will be interesting. The something unfamiliar could be a novel goal stimulus or an unfamiliar trigger-goal sequence. Mind you, the novel goal stimulus will also cause an unfamiliar trigger-goal sequence.
Thus the reason for doing it again could be either the goal stimulus is interesting or the trigger-goal sequence is interesting. This gets back to the concept of a redo interest on the action habit that I got rid of on the 25th Oct 2010.
Conscious stimuli
Do I really need the idea of subconscious stimuli? Why can't a stimulus (or sequence) be novel (interesting because it is unfamiliar) the first time it is experienced and then become neutral when it occurs a second time? They then are used as triggers for action habits when they repeat. Then if it occurs again the action habits are searched for any worth repeating because the goal is interesting or trigger-goal sequence is interesting. This would need all the partial sequences generated on STM to be candidates for 'if it occurs again'. Thus the sequence ABC would be in STM as C, BC and ABC. They would all be conscious. They would form the current stimulus list. Then should anyone of them have an action habit with an interesting goal or potential to form an interesting sequence it will become the attended to one.
3rd May 2011 ABABAB
When to make the 1st response? On 20th Jan 2009 the second AB reduced the interest to neutral "because it was unexpected" and the 1st response is after the 3rd A. On 4th Jan 2010 the 1st response occurs after the 3rd AB rather than the 3rd A due to interest in the longest sequence. On the 25th Jan 2010 I argue for responding after the 3rd AB because even though we are bored after the 2nd AB we want to be slower to react to allow thinking to occur and make sure that the boredom is repeated. On the 15th Mar 2011 I wrote “We still have to retain the idea of the longest non-repeating sequence as the unit of conscious responding so that only after the third AB in the sequence ABABAB, do we perform a reflexive response. It is only after the end of the second AB that we have an expectation of what follows AB and can predict this. It is when the prediction of what one can expect next is realized that we are bored." This statement does not consider the immediate response due to a repeat of the stimulus on the 2nd AB. When the second AB occurs we have a sensory repetition and an expectation of a repeat according to the experience at that point. This all seems to imply we should react after the 1st repetition, which is after the 2nd AB.
The second feature that is currently implemented is to wait (orient response) for a second repeat before trying another reflexive response. This is currently implemented as emptying the STM when a reflexive response is performed and therefore it has no stimulus in it to detect a repetition. The reasoning for this wait / orient response between reflexive responses was documented on the 11th Feb 2011. [Read 9th Feb 2011 too]
Repeating Action
I used to have the idea that an action habit is repeated to make sure it has been learnt and it can be repeated. I.e. check that the rules of the environment have not changed. And then the habit can be done subconsciously. This repeat of the action habit may be natural due to the new trigger-goal stimulus sequence that the action has produced even when the goal is familiar. Then when the trigger reoccurs, the trigger-goal sequence is familiar and the response is not produced. It only gets reproduced when the goal is interesting. This works in with the proper definition of boredom. It is not just the repeat of a stimulus. That only produces a neutral interest in the stimulus. Boredom results from thinking one-step ahead. That is, recalling a trigger and investigating if there are any interesting goals or interesting trigger - goal sequences. If no, then we are bored using a one-step look-ahead thinking. Then we produce a reflexive response. If we had a two or more look-ahead thinking process then we would have a lower chance of being bored because of the chances of finding an interesting goal or sequence to perform. So a no look-ahead thinking process would be purely reactive upon every stimulus repetition.
Habits
This leads me to think that a habit is always a trigger-response-goal pattern. But when the response is being done the trigger-orient-goal habit (S-habit) is being recognized (also being done) (self-entertainment). The trigger-response-goal habits in which the response is a non-orient response (A-habits) do not have interest. Only raw stimuli and S-habits have familiarity and interest. Therefore if an A-habit is to be repeated when the trigger-goal sequence is interesting somehow the S-habit produced by the A-habit must be linked to the A-habit. Can this be done dynamically at run time? For example, an A-habit has been done and now the trigger stimulus occurs. If the goal of a related A-Habit is interesting then the A-habit is started. If any trigger-goal S-habit is interesting then use its goal stimulus to find any action habit that produces it and start it.
4th May 2011 Mirror Neurons
Mirror neurons are the S-habits that recognize the sequence of stimuli composed of a trigger and goal stimulus independent of whether one causes it personally through action or perceives someone else causing it.
5th May 2011 AAAAA
The reason for waiting for the 3rd AB is to confirm that AB is really followed by AB. It is this wait for confirmation feeling. Does the world really work this way? It is equivalent to saying; "I have experienced ABAB. This is a repeat of AB. The repeat is new / novel / unfamiliar. Let me repeat that action (orienting) to see if it is true." Is this the same as the wait before trying an action response a second time immediately after it has just been done? It's like giving the world a chance to change after the 1st response. It is as though the trigger-goal sequence and/or the trigger-response-goal sequence is novel / unfamiliar and this is what is attended to. Being new it has no associated next response or stimulus and thus we wait and perform an orient response. The sequence is the interesting thing grabbing attention. But this means that I need to produce the AB-AB sequence as unfamiliar so it is interesting. And then I must recognize the repeat and remove it from STM and just leave the last AB. It also means the AB-response-AB must be seen as a new unfamiliar sequence that is interesting. [Re-read 3rd May, and 9th Feb 2011 too]
Structure
What I need is a new structure for habits in general. I have the idea of an acton being a trigger stimulus and response. I have the idea of a stimulus containing a where part that could be the response to do to get the goal stimulus. And the where-part could be the orient response for a stimulus. So an experienced stimulus could be response-stimulus. And an action could be a stimulus-response. When you pay attention you use the where part (orient response part) of the experienced stimulus. When you think you use the response part of the goal stimulus. When you respond do you use the response part of the goal stimulus or the response part of the action? The challenge is that one trigger has many possible responses each of which could have many goals. Could the response part of this structure be represented by the expected muscle stimulus? In other words every response generates a stimulus from the device as pure response feedback. This is the pure muscle tension kinesthetic feedback. It is not joint angle kinesthetic feedback. When we first move our muscles in the womb they generate these feedback stimuli. Could the expectation of them act for them as the response where part of a goal stimulus?
30th June 2011
See the 2011 Scientific Research and Experimental Development (SR&ED) tax credit claim.
Details about the Canadian Revenue Agency’s SR&ED tax incentive program are here.
11th July 2011 Actons
I’ve been reading about PID Controllers, feedback control, oscillators and servomechanisms. I think an acton should be given a command value and a goal stimulus. At the lowest level there are devices which are sent command values from lowest level actons to tell them what to do. It might be a power output value, a volume value or speed etc. The device then performs its action based on this command value. However there are sensors reading the results of these devices. The lowest level control acton should continue to send the command to the device until the goal stimulus is achieved. In controller terminology the goal stimulus is the set point. But in a PID controller you don’t send it a command value. It decides itself what command value to use, dynamically modified by the controller to reach the goal. But you do send the set point to a specific controller and therefore a specific device. This analogy would mean that each acton is identified like a device and only given a goal stimulus to reach. Each lowest level acton would have the capability of controlling only one device and only on a linear scale. And it would be given a lowest level sensor reading to achieve.
14th July 2011 Actons
The lowest level control component of intelligent action must be given a command to start and a goal to achieve. This must be cued by the recognition of a trigger stimulus and the desire / interest to achieve the goal. The absolute lowest level must be a device command to make a change. This absolute lowest level has no intelligence and is just the device. The level above this must be the control component given a command to start doing its thing and a goal to reach. It must thus have a degree of intelligence that knows what the current stimulus is, what the provided expected stimulus (goal) is and what command it needs to give the device to achieve or try to achieve the goal. If this is a PID controller then the goal is the set point. It must be able to read the same senses & sensors as used in the goal stimulus and do an error/difference comparison with the values in the goal stimulus. It can use this to determine the P (Proportional) part of the command to the device. However, for the I (Integral) and D (Derivative) parts the controller needs to have some history of previous errors / differences. So maybe the control component needs to keep trying to reach the goal if the 1st proportional command does not do so. Each time it tries is a cycle and it accumulates the I and D parts to gradually converge on the goal. So inherent in an acton (control component) is the idea that it loops until the goal / set point is reached. But if the goal is not reached (maybe in an expected time or number of cycles) then an unexpected difference is experienced and this may attract attention so learning can take place.
At a higher level there must be a way of sequencing two actons such that when the 1st reaches its goal the second is started to achieve its goal. Thus there must be an acton A (control component) that sends commands to two actons B & C in sequence. If the second acton C is the same as the first B then one could have a loop which returns the component back to its original setting. If A by default repeats until it reaches its goal then B & C may loop many times. I don’t yet know what A’s goal might be.
What about controlling 2 or more devices in parallel to achieve a goal? Having discovered B and C in sequence achieves a goal and knowing that the devices controlled by B & C are independent it could try doing B & C in parallel instead of sequentially.
The goal stimuli that change when a particular lowest level acton is performed would have to be discovered by performing random actions. This would accumulate experiences of what actons to use to achieve certain goals. The subset of stimuli changes that consistently occur for a given acton would be the ones that it could be commanded to achieve. Once these have been subtracted from all changes the remaining changes can be higher level goals of higher level actons.
16th July 2011 Actons in parallel
Each acton should be able to activate a pair of actons at the next level down either in parallel or in sequence. Read 20th March 2011. When an acton is activated if the set point is already the same as the actual stimulus the acton does nothing. But if an external factor changes the stimulus while the acton is active, the acton will react and try to achieve its goal stimulus again. Thus at any one time many actons could be active but not be doing anything because they have reached (are at) their goal. If the first one in a sequence of two actons is already at its goal stimulus then just the second one gets done. How do active actons get deactivated?? The activation request (signal) could be continuous while the action is necessary and as soon as it is turned off, presumably by the higher level goal being reached, then it no longer receives its activation signal. This means an acton is continuously sending its set point goal request to the two actons it commands. When the two actons being activated are sequential the completion of the first must be necessary for the activation of the second because you don’t want the second to become active prematurely.
Maybe the next experiment I do should be to incorporate the STM pattern recognition algorithm into a PID controller to see if it can do the proportional, integral and derivative calculations.
Distributed STMs
What will most likely happen if I incorporate the STM pattern recognition into a PID controller is that each acton will have its own STM. So, just as I have distributed the ‘where’ and ‘what’ information into every level of the recognition hierarchy I will be distributing the STM to every level. So should I be concentrating on incorporating all aspects of learning into each acton?? Each acton becomes capable of adapting. But it must have some constraints to do this; it can’t be all knowing. At the lowest level it would be constrained to control a single device. It may also be restricted to a specific ‘where’ to monitor as its set point goal. This in fact is what is contained in its goal stimulus. If it is executing subconsciously and recognizing its expected stimuli then these stimuli are effectively habituated and don’t reach consciousness. Only stimuli that change from a ‘where’ that has no active acton interrupt consciousness. This gets back to the concept of multiple active habits recognizing expected stimuli so that the unexpected attract attention.
22nd July 2011 Actons are self-managing
The lowest level acton will also have to be self-managing as in they are given an expected stimulus and they control their device to maintain this feedback value. Thus every device has a sensor attached that reads its setting. An outside perturbation may change the setting and the acton must adjust itself to bring the setting back to the expected stimulus value. The identity of the acton for the device is the same as the identity of its sensor which is the 'where' part of the sensor. Thus a goal stimulus does contain the identity of the actons to be activated in the 'where' part of the stimulus. [This confirms the idea of the last sentence from the structure paragraph of the 5th May 2011]. Thus given a P-habit goal stimulus the pattern of actons to be activated (Gallistel uses the term potentiated) is contained in the 'where' part. Given a sequence of stimuli as a goal the temporal pattern of 'where's provides the temporal pattern of device activation. All actons and thus devices remain active until they are given a new goal. Thus all devices are always active with some goal. When two devices have an effect on each other and are given new set points/goals in parallel the change of one to reach its goal will disturb the other. But the other will dynamically adjust itself to maintain its trajectory to its set point/goal.
Where information
There is also 'where' information attached to stimuli that do not correspond to a pattern of device sensors. Colour, volume and frequency are examples. If the 'where' information in a goal stimulus corresponds to a device then the device is activated to achieve the goal. If the 'where' information in a goal stimulus does not correspond to a pattern of device sensors then it is used to activate attention on the appropriate pattern of stimuli. This is the orienting response that does not require a set point / goal given to a device!
Attracting Attention
A device that achieves its goal effectively neutralises interest in its goal such that it does not attract attention. A disturbance to the device sensor results in an unexpected change that the device should be able to handle and not make conscious. These ones are the result of what it has learnt to handle. It will be a change beyond the ones it has learnt to handle that will attract attention.
Responses
So responses are just the 'where' part of a goal stimulus. And if all devices have a sensor that they use to match the goal's 'what' part then how do we formulate actons in which the goal contains a stimulus that is not measured by the device's sensor? In other words, the goal is one from a non-device / kinesthetic sense. Obviously a change on this sense must have been detected when an action was performed. The change on the device's sense would not have attracted attention. But it would have occurred in parallel with the device's stimulus. If there are no responses between stimuli then when a random action is performed in response to boredom the response is found in the where part of the device's sensor reading/stimulus that is received in parallel with the non-device's stimulus. This then becomes the goal stimulus should the non-device's stimulus become interesting and desired.
23rd July 2011 No-Response
I will also need the devices to have a resting / neutral stimulus position corresponding to the no-response. This is because if a device is told to move left it will continue moving in that direction until it is told to stop with this resting goal. [Read 8th Jan 2011]. But this might not be true for a device that takes discrete / symbolic stimuli as goals.
Devices and Actons
I need to clarify my terminology. A device is the lowest level mechanism that produces action. It has a sensor attached to it that reads its setting. The lowest level acton (level-1) is a controller that sends signals to a single device and monitors the device's sensor readings. Actons learn. They are given the goal to reach and learn which signals to send to the device to reach it.
Discrete and Graduated devices
A device that moves left, right or stops, has 3 commands it can be given. It is a device with a discrete / symbolic sensor that returns one of these three possible readings. I'll call this a discrete device. A device that can be given a graduated value such as rotational angle has a graduated (and circular) graduated sensor. I will call this a graduated device. The acton for a discrete device will send one of the discrete commands to the device. If it is the Left, Right, Stop device and it is told Left then it will continue to move left on each cycle. Its sensor's returned reading will be 'left' unless it hits a wall. Then it will return the stop reading. This is when the acton must wait for a command from higher up as to what to do. So it makes the stop reading available for a higher level to learn what to do. Or should the device just move left once and hold that position. If it fails to move left because there is a wall it does not achieve its set point.
If it is a graduated device then when its acton is given a goal value it will read the current sensor reading and send a value to the device to move it based on the difference between the current reading and the goal reading. It will continue cycling using its STM as a PID calculator to adjust the device so its sensor reaches the goal/set point. It will then report any change to this sensor reading caused by an external disturbance to the acton at the next level up. Does this acton level-1 need to learn? If there are some reflexive actions that continue to be successful maybe it should learn them. Such as if it has been told to stop but it gets bumped to the left should it do a single right action to compensate?
Higher level Actons
At the next level up and higher (level-2+ actons) an acton must similarly be given a goal and try to achieve it. But it must incorporate learning because it has to determine what to do when level-1 actons are disturbed. It has the possibility of doing two actons in series or in parallel. The goal it is given may not be composed of the goals it gives to its two subordinates. It also must be able to repeat itself until its goal is reached. It would seem reasonable that the goal given to a level-1 acton would only contain a stimulus with the devices sensor as the 'where'. But at level 2 the goal provided might be based on non-kinesthetic sensors.
When a discrete device with sense #1 has just moved left and the input sense #2 reads a B then the P-habit perceived is 1-Left & 2-B. If it now moves left again and hits a wall the P-habit perceived is 1-Stop & 2-B since the position has not changed.
No device feedback
So now the question is can Adaptron behave with interest in a body that does not give back any kinesthetic stimuli? The last response sent to the device must be used in combination with the perceived stimuli. This is the only place the responses to be done can be stored. That is it must be stored as a device's sensor stimulus in parallel with the perceived stimulus. It also assumes the devices never fail to perform the required response to get their desired goals. It also never detects any external disturbance or compensates for it. Thus the level-1 actons always return the goal stimulus as reached successfully.
No-Response
The no-response / orienting response does not send any requests to any devices. Contrary to what I wrote this morning all devices are held in a static position until they get a new set point goal. A discrete device told to go left does not continue to go left. It just does it once and stops. The same is true for a graduated device. It continues to hold its position. So do I need a stop or resting goal to combine with the perceived stimulus when performing an orienting response? No, I don't think so. The goal stimulus just does not contain a device stimulus part. The device sensor would still be providing the goal reading but since it has not changed it would be habituated. Thus no "stop" response is required.
Active A-habits
An active A-Habit is really an active S-Habit in which there is a device stimulus as part of the goal. All A-habits that have been started represent an acton and they will remain active until they fail to get their expected goal. This can occur because a more recent A-Habit has taken over the devices they were using. The failed A-habits don't complain because the new goals reached should be recognized by the new A-habits. Any unexpected goal from a new A-habit will be recognized because it will be unfamiliar.
24th July 2011 Trigger in Actons
An acton must also have a trigger that it recognizes. When I start an A-Habit it activates its two actons. But the actons may also need their own trigger stimulus for them to start executing. Thus an acton can remain activated but not execute unless the appropriate circumstance arises. The first thing an acton does once it is activated is to test the current value of its sensor to detect how far away it is from the given goal. This is just like a TOTE unit. If it is a PID controller it needs this information to determine an error value. If it is a 'walk to the kitchen' acton it needs to check the current state to determine what to do first - 'get out of the chair' or 'turn around'. In the middle of executing 'walk to the kitchen' it must handle any obstacles and 'step over' them.
26th July 2011 Habit execution
Habits are started because they have a goal stimulus that is wanted because it is interesting. Interesting because it has only occurred once before and we want to make it familiar. So as a habit executes it pays attention to the 'where's of the goal stimuli. If the goal includes a device stimulus then it has sent the expected value of the device stimulus to the device in the previous step expecting to get it back as a stimulus in this step. When it does not get what it expected it fails and stops. At this point an unexpected sequence has occurred because some other stimulus was obtained instead. This is based on only expecting ones with an interesting goal (the active habits). If this sequence is new it will be interesting and attract attention. If it is familiar it will become the most recent version of what to expect. [Read 15th Dec 2010 for recency argument, 13th Aug 2010]. If it gets what it expected, it continues. If it continues to the point where the interesting goal is accomplished then this goal stimulus becomes familiar and the habit succeeded and stops. But do I need active habits for all expectations even if they don't have some interesting goal? Do I need them for all recognition habits too?
I have an STM per sensor and then combine them within each sense. Then I combine the senses to form P-Habits that also have STMs. This is all to do with recognizing repetitions and sequences. Any new one will be interesting. Do I need an STM per binon? Each one that is recognized has a list of possible expectations of what could follow it. When one is recognized all the next ones must be expected. If next ones are interesting then the action part (if any) of them will be done. But only one current stimulus is attended to that is used to determine what to do next. And only one of its expectations is started. Are all the others still active until they fail?
A habit that has been made active that must wait for its trigger stimulus to occur before being executed must be deactivated. Would this take place when a different habit is started with the same trigger stimulus?
27th July 2011 Sequences
There appears to be two kinds of sequences. There are the STM created ones which are done per sensor / sense and relate things at the same level of complexity (S-Level) and there are the ones that attention is paid to. The latter are the habits we execute with expected goals. The trigger and goal do not need to be at the same S or P-Level or from the same senses / sensors. They also do not need a device stimulus in the goal. But they do form a tree structure such that we can have sub-goals. The tree can consist of parts that need to be done in sequence or in parallel. The former sequences are just recognition sequences that can be used to detect repeats and recognize longest sequences that can be attended to. They have the trigger and goal from the same sense / sensor combination.
And when you are self-entertaining you are executing the latter sequence tree while recognizing the former. When executing the learnt habit sequences and we get to the point where we need to recognize a STM recognized sequence this must be matched too. [Read Goal State 30th April 2011]
Further thoughts about what is used as the goal of a reflexive response. We are trying to avoid a repeat of the trigger that caused the reflexive response. The first stimulus after the response that is different would not be the accepted goal. It should be the longest unfamiliar (interesting) sequence after the response.
28th July 2011 Action Habit structure
I think the following structure will provide all the properties required for action habits. An A-Habit will be a sequence of one trigger stimulus St and one goal stimulus Sg. When the action habit is enabled (is active) and the trigger stimulus is recognized the Sg will be done. A goal stimulus Sg is the combination of two A-habits A1 and A2. When the Sg is done its two action habits are simultaneously enabled. But being enabled does not necessarily mean they will be done. Their trigger stimuli must be recognized. Thus A1 and A2 could be done in parallel if both their trigger stimuli are recognized when they are enabled. They could be done in sequence if the performance of A1 establishes the trigger for A2. They could be done any time later as long as they stay enabled and their trigger stimulus occurs. Thus they may not get done at all before they are disabled. They may be done cyclically if the St of A2 is the same as the Sg of A1 and the St of A1 is the same as the Sg of A2.
Now to repeat a question from the 16th July, how does an enabled action habit get disabled?
The part of all the stimuli that you get instead of what was expected attracts attention. Effectively all the stimuli from all senses / sensors are expected by enabled habits. Any stimuli or part thereof that does not match a Sg becomes conscious so it can be learnt about. A part of a stimulus pattern that is not perceived, is missing, shows up as a different stimulus value. Therefore when you don't get what you expected you still get something instead. The missing stimulus does not attract attention. The different one that you obtained instead attracts attention.
30th July 2011 Action Structure
On 9th Oct 2010 I defined Acton for the first time. I now have a new definition and a new structure for it. A trigger stimulus is St. A goal stimulus is Sg. Sg contains two actons, Aa & Ab. An Acton A = St -> Sg where the arrow implies time / order of occurrence. When St becomes conscious and is attended to this enables the acton's Aa and Ab in parallel. When an Acton is enabled it immediately checks to see if its St has occurred. If it has, it enables its Aa and Ab, which likewise check their St's recursively down the tree of actons. When the Sg of an acton is recognized the Aa and Ab are disabled.
At the lowest level there is an acton A1. Its St could be an Sk & Ss where Sk = a kinesthetic stimulus and Ss is a sense stimulus (non-kinesthetic). All stimuli are comprised of Sw and Sv where Sw = sense/sensor location (ID) - where information and Sv = sensor value - the what information. At the lowest level the A1 has a Sg that could be an Sk only (i.e. the Sw and Sv are both kinesthetic). The lowest level acton Al = Skt -> Skg or a tree of Al's. If an Al is enabled when its Skt is recognized then the Skg is sent to the device. When Skg is recognized from the device sensor the Al is still enabled. This Skg may be recognized as a Skt of another enabled Al. If the Skg does not occur then a Ski will be recognized instead. This Ski may be recognized as a Skt of another enabled A1. A lowest level acton Al will remain enabled until the Sg of its enabler acton at the level above is reached.
Actons at higher levels than the lower level Ah = St -> Sg, or maybe Ah = St -> Aa & Ab -> Sg, or maybe Ah = St -> An -> Sg where An = next acton and An = Aa & Ab. But we have an acton disabling its Aa & Ab when the Sg is reached. What if the Sg is not reached? What disables the Aa and Ab? How can an Acton recognize when its Sg is not reached? The Sg could be a sequence of stimuli. As soon as its sequence fails then the acton fails but remains enabled. When it fails it should disable its Aa & Ab.
If Adaptron is using symbolic input and is on a square labelled D and it goes left to a square labelled E this will be captured in the following structure. The moving device will have two commands L for left and H for halt. Its sensor will return these kinesthetic values after it has done them. The move left action is going to be the combination of two actons. Aa = H(St) -> L(Sg) and Ab = L(St) -> H(Sg). I could put an An between the St and Sg where the An is the most primitive response. Then Aa = H(St) -> L(An) -> L(Sg) etc. Now the experience that moves us left from D to E is: H&D(St) -> Am -> H&E(Sg). Where Am = Aa & Ab
But how does it learn the Aa=H->L->L without the D and E as part of the stimuli? Is this what a baby is doing in the womb? Or can this be extracted from multiple experiences involving additional non-kinesthetic sense stimuli?
New habits that are enabled and performed should override the commands to devices that are being controlled by already enabled actons such that the previous actons will fail.
I will need to mark all stimuli as unexpected when received and then mark them expected as the actons get their Sg's or parts thereof. Then the remaining unexpected stimuli will attract attention. These unexpected stimuli will have never been Sg's of actons. They may however have been familiar because the STM created them.
STM continues in the background recognizing sequences of stimuli and the actons are always comparing their St's and Sg's with STM values.
31st July 2011 Reaching Goals
What if the Sg of a high level acton does not contain a sequence of all the stimuli that occurred since the St was recognized? This should be the case. The Sg is just the expected stimulus. If the Sg is not recognized, how long does the acton stay enabled? At the lowest level the Sg is immediately tested after the device command is issued. This signals that this acton has reached its goal. Does this information go back up the enabled acton tree so actons can then test their goal stimulus? This would be a complex process considering that some Aa & Ab pairs are done in parallel and others in sequence. It would require the rule that every master acton has at least one of its two slave actons that is sequential such that when the slave has reached its goal the master can check its goal. If both slaves are sequential then the second slave must have done its job. If the two slaves are parallel then the first must contain the sequence of actons and be used to signal completion and checking of the master's Sg. The use of parallel actons is necessary to create contingency actons that kick in when something unusual happens. E.g. we encounter a closed door on the way to the bedroom.
If we make the rule that the second slave Ab must signal completion for the masters Sg to be tested then actons to be performed in parallel could always be assigned to Aa. Thus the Ab side of the tree always contains the sequential actons. So when an Ab succeeds because its Sg is recognized the master is notified. Then the master can check its Sg. If an Aa succeeds because its Sg is recognized then it is re-enabled by its master so that it can handle its parallel situation again if it should arise. Thus any looping acton must be placed as an Aa slave so it gets re-enabled until the Ab recognizes the event that will terminate the loop. When the master checks its Sg and it succeeds or fails it disables its Aa and itself before notifying its master.
1st Aug 2011 Partly familiar
If you see something which is part of a larger thing and you see it from a totally new angle in a new light setting theoretically you have never seen this exact image before. However it does not attract your attention. This is because it is recognized as part of something already familiar. From a new angle but it retains its internal relative proportions and allows you to identify it as part of something already known. In a new light setting but its relative brightness to those things around it is familiar.
Unexpected = Novel
Any stimulus that is novel will be unexpected because it has never occurred before. And familiar stimulus that is unexpected causes a sequence that is unexpected. This sequence is novel. A repeated stimulus can be unexpected on its second occurrence. Change of a stimulus on a sensor does not mean we should pay attention to it. It is the unexpected changes that attract attention. And it should only be the STM that decides if a sequence is new and thus unexpected. This would mean that STM is creating all sequences even across actions, between trigger and goal. We also have to have action or STM habits that expect a repeat and do not cause a reflexive response. The repeat not only has to be expected but also attended to before this can happen. So it is only when there are no unexpected stimuli that a repeat will attract attention.
Music - rewarding
Why is it that we enjoy listening to the same piece of recorded music more than twice? The first time it was heard it was interesting and the second time it was familiar. It must have been the interest from the 1st hearing that generated a pleasure that is regenerated upon hearing it subsequent times.
2nd Aug 2011 Combining Actons
Maybe all pairs of actons, which are done as a result of attention, are combined sequentially all the time. Thus the pairs are continuously being formed rather than only when the second goal is interesting.
4th Aug 2011 Sg not required
I don't think I need the Sg to be checked when performing A-Habits. Each acton only gets done when its St is matched. When it is done whether it succeeds or not it returns to its master. If it is the master's Aa then it gets re-enabled. If it is the master's Ab the master returns. Any failure is either recognized by another acton running in parallel or is unexpected and needs to be learnt. Sg is used for learning but not for execution.
29th Aug 2011 Unexpected
So the AAA sequence gets a repeat at position 2 but it is unexpected so no reaction occurs. The 3rd A is expected and the reflexive response occurs. Note that a familiar stimulus can be unexpected. In this case the sequence with the familiar stimulus in it is novel. If ABC is familiar and BCD is familiar but ABCD is a novel sequence then is BCD unexpected or is just D unexpected? I feel that just D is unexpected and this is what we pay attention to.
What about unexpected parts of a scene that all occur simultaneously? You are shown a photo of a street scene with cars and pedestrians but there is a desk in the middle of the road. The unexpected item (the desk) attracts attention. If the photo is of an office floor with many desks and workers and there is a car in the middle the car is unexpected and attracts attention. Is it the smaller familiar part that attracts attention?
3rd Sept 2011 Sequential Expectations
In the above analysis of ABCD I felt that the D was unexpected. But actually it is not. It is the sequence that is unexpected. The relationship or change between ABC and BCD (which is ABCD) is the unexpected thing. If both A and B are familiar but AB is unfamiliar when AB occurs it is the AB sequence that is unexpected.
Unexpected sequence means the sequence is unfamiliar, i.e. novel. But when a stimulus repeats it collapses to the stimulus. So the second A in AA is familiar but the AA is unexpected. I then say the 'A' is unexpected. But if I keep the AA and use this as the unexpected / unfamiliar stimulus, the 3rd A will cause a repeat of AA and AAA will be unexpected / unfamiliar ad infinitum.
And the same applies to a scene in which two parts have never occurred together before. Neither part is unexpected. It is the combination that is unexpected / unfamiliar. What if the scene contains WXYZ, W and Z are familiar but X and Y are novel? XY is novel and W...Z is familiar. But the biggest unexpected / unfamiliar combination is WXYZ. Does it attract attention? No the biggest novel combination XY attracts attention. The rule should be:
Given A and B,
A B AB
Novel Novel Novel
Familiar Familiar Familiar
Novel Familiar don’t combine
Familiar Novel don’t combine
Applying the same rule to sequences means that novel stimuli are combined into the longest novel sequence. If the longest sequence so far is novel and a familiar stimulus occurs then don't combine it with any part of the previous sequence. And if the longest sequence so far is familiar and a novel stimulus occurs then don't combine it with any part of the previous sequence. But combine familiar with familiar until the longest is built or until a novel sequence is recognized.
Because stimuli are processes at the same level of complexity before combining them at the next higher level, in the case of parallel stimuli it would be possible to miss a novel stimulus at level-2. The novel level-2 stimulus could be made from the combination of two familiar level 1 stimuli but a level 3 stimulus could be novel formed from two other level 2 familiar stimuli.
My STM algorithm is quite complex due to it being used to check for repeats but also generating long sequences up to and including the repeat just so I can determine if the repeat is expected or not. But maybe expectations should be created by the main loop when thinking is turned on and then recognized if the goal is as expected. Then if it is a repeat reflexive action should be done. This would make STM simpler for recognizing repeats and it would always flag them as familiar. If thinking were disabled reaction would occur after the second A in AA and if thinking were turned on it would happen after the third A in AAA.
8th Sept 2011 Acton formation
The fact that what happens next (orienting) when no action is done, may be more than one valid goal stimulus would suggest that actons not be used to learn orienting responses (the do nothing variety). But then the question is how does an active acton do the waiting for an expected goal to complete when that goal is an S-Habit? There is no problem for the trigger stimulus being a sequence because the acton does not execute until it is perceived. The solution is that the acton when executed does not check for goals. It just does the action and terminates. It has no need to know whether it succeeded or failed. This is the job of consciousness / learning actions. Read 4th Aug 2011.
When we are trying to achieve a goal, an unexpected stimulus occurs (one with no acton as a trigger to handle it) and we try a response then this new learnt action habit must be added as an Aa slave. It is to be done in parallel should this anomaly occur again in this context.
14th Sept 2011 Most Interesting
I am trying to refine what stimuli combinations are most and least interesting. I have decided that a repeat either in parallel or sequence is less interesting than a familiar (non-repeating) stimulus. Thus there really are four levels of interest so far. - Interesting, Expected Interest, Neutral, and Repeat.
I have also decided that when forming a P-Habit that is many levels high from all familiar primitive stimuli that the highest novel combination is most interesting. For example, given the binon structure in which D, E and F are primitive at level 1. B and C are level 2 and A is level 3.
A
/ \
B C
/ \ / \
D E F
What if D and E are familiar, B=(D^E) is novel and F is novel? C should not be formed because it would combine a familiar with novel stimulus and thus A would not be formed. So what attracts attention, the novel B or F? According to the principle B as the highest level novel stimulus should attract attention. A hopefully more realistic example is when you are looking at pictures of weird combinations of things. You are shown an office scene with a yellow car sitting in the middle. On top of the car is a small bonsai tree. The tree is quite unique because there are leaves all around the bottom part of the trunk and the branches sticking out above are bare. All the parts making up the office are familiar. All the parts making up the car are familiar. All the parts making up the tree are familiar. Attention is first drawn to the novel combination of office and car. This combination becomes familiar. It is at the highest level of complexity of parts. Then attention is drawn to the tree on the car. After this is familiar then attention is drawn to the novel combination of leaves and branches.
What if D, E, and F are all familiar but both B=(D^E) and C=(E^F) are novel? According to the same thing happening in sequence in my Test Run-1 12th Sept 2011 I would combine them to form the novel A=(B^C).
The current sequential rule to only combine stimuli with the same interest level stops combining familiar stimuli when the first novel one is found at any sequential level (level M for example). And then it continues combining novel ones at this level (M) comprised of familiar ones below. Then these novel ones are combined to higher level novel ones. But as soon as a familiar one happens in the sequence at level M this becomes the one that attracts attention. It can't be combined with the previous novel one at level M.
Extending the concept that parts only get combined when they are of equal novelty means that repeated stimuli should not be combined with novel or neutral/familiar ones. When recognizing stimuli from a sense with dependent sensors at the lowest level I combine adjacent values if they are the same i.e. they repeat. This needs to be done at all levels as I do for S-habits using my STM algorithm. But does it apply to independent sensors or across senses (that are independent) such as P-Habits?
When you have the same value on two independent sources then it is just another combination / pattern that must be recognized. It is not a repetition that is uninteresting. But should a pattern be formed across independent sources from parts that have different novelty? No the parts should have the same novelty to be combined. This should also apply to the same repeated status.
17th Sept 2011 Directed Attention
With the new acton structure for A-habits and the current attention attraction algorithm I ask where does it direct its attention as when we are concentrating? Obviously directed attention is conscious focus of attention on a particular "where" based on a goal stimulus. But it could also be the focused attention on the St of the Ab second acton. Either way it is the conscious execution of a known habit. It is the thing we do when we are practicing a habit. Why do we practice a habit? If the stimuli are all symbolic then there does not seem to be a need to practice them. Or is it when we first repeat a newly discovered action habit (because the goal is interesting) that we are practicing and concentrating hoping to get the goal? Then when the habit has been proven (familiar goal) the action habit is done subconsciously? With graduated stimuli I can see the need to practice, concentrate and direct ones attention because the success of the habit can be subtly dependent on the exact value of a response. OK - I lean towards the idea that we direct our attention on the goal stimulus when we are doing the first repeat of an action habit because we want to achieve the interesting goal. Probably based on the success or not of this habit we incorporate it into its master action habit. The principle for processing expected goals from 30th April 2011 now becomes important.
18th Sept 2011 Concentration
When we are directing our attention we are concentrating. The goal was wanted or was to be avoided. If we are trying to visually find something we are holding the "where" focus for some time while the habit gets executed. But during a longer habit such as "get a glass of water" our concentration goes back to neutral and we can spend time thinking. What are the other criteria for maintaining directed attention? Is it something about the habit being novel or familiar as well as the goal's novelty? If a reflexive response gets a familiar goal and forms a new habit is there an interest in performing the habit again when its trigger occurs? It is only when we are concentrating on performing a habit (practicing) that we get the success or failure feeling that is used to "cement" the habit. Read 11th Dec 2010. Which brings me back to the idea of an action habit having an interest level that is neutralized to familiar when it is done the second time successfully.
20th Sept 2011 Shortest novel sequence
Based on today's 1st test run it appears that across multiple independent senses (or sensors) the shortest novel sequence attracts attention before a longer novel sequence on a different sense. One would also expect a shorter repeating sequence would attract attention before a longer duration repeat on another sense. This appears to be the opposite of the rule between sensors that are dependent.
But the shortest repeat attracting attention as in test run-3 doesn't seem immediately obvious. We are expecting these repeats to habituate. We no longer pay attention to them because they repeat. But if the repetition is the only thing left to pay attention to it is boring.
21st Sept 2011 Practice mode
When a reflexive response is performed the unwanted goal (a repeat of the trigger stimulus) causes us to have a concentration level of "uninteresting". This allows any other uninteresting, familiar or interesting stimulus to attract attention (distraction). When a response is done because the expected goal is interesting (was novel) we have a concentration level of "interesting". This allows another interesting stimulus to attract attention (distraction). Should such distraction / interrupting stimuli become the new goal for the action habit being practiced? Or should only stimuli that don't match the goal expected be used as new goals because they caused the action to fail? I think both distracting and mismatch stimuli should be used as the new goal for a habit being practiced. If the habit is not being practiced what do we do with such stimuli?
Can a habit not be in the practice mode? If I am going to get a glass of water the step over obstacles and get out of chair habits all become active. They are being done subconsciously and we would normally think of them as not being practiced. But if I have a problem getting out of the chair because my pocket gets caught on the armrest and I try a manoeuvre that succeeds and corrects the problem then this newly learnt action habit is added to the get out of chair action habit. So this would imply that the active habits are ready to be corrected and added to at any time. It also makes me think that the A-Habit is really a trigger - goal pair with a list of possible actons that could be used, the most recent one being the one to reuse. This is different from the current model that says it is a trigger with a list of responses and what goals will be obtained.
Now the answer to the question about distracting and mismatching stimuli is answered. The original action habit does not change. A new one is formed based on the trigger and new goal.
24th Sept 2011 Distraction
Will an attended to / conscious stimulus of equal interest to the current goal stimulus distract one from the current action habit? Does the current action habit then get disabled and replaced with the new one? This is often true when the goal has an expected interest. In this case the goal has been thought about. Example 1: A man puts down the jam he is intending to take with him when he leaves. He then goes and checks the bathroom. He leaves but forgets to take the jam. Example 2: The lady knew she needed a photocopy of her passport with her when she went on a trip. She made the copy and took it with her but left the passport in the copier. The second example may be an action habit in which the primary goal was achieved before the correct end of a well-formed habit. The copy was made and this completed the action since what was wanted was obtained. More practice was needed to add the ‘remove original from copier’ such that it was a subconscious part of the habit ‘make a copy’. The first one is an example of an active action habit that was replaced by another at the same level of interest. However the goal of leaving was still uppermost in his mind.
30th Sept 2011 Generalization
Generalization occurs when one part or the other part of a P-Habit is used instead of the whole. Parts correspond to type attributes because the two parts are independent and an OR relationship exists between them. When two parts are dependent the relationship is AND. If two dependent sensors are adjacent to each other the relative brightness is what is important. They cannot be used independently for generalization. However the brightness/contrast pattern of an object is independent of its shape so one can generalize independently along these two dimensions or features.
When one has 3 independent parts of a P-Habit one would not want to generalize based on just one of the parts. There is not enough similarity to the original 3-part stimulus. But generalizing on any combination of 2 parts would seem reasonable. A rewarding experience with just 2 parts would then generalize to either of its single parts.
When independent senses or parts are combined the number of combinations at the different levels of complexity (number of parts involved) is equal to 2^P-1. Thus with 2 parts the 3 possibilities are A, B and AB. With 3 parts the 7 possibilities are, A, B, C, AB, AC, BC, and ABC. With 4 parts the 15 possibilities are: A, B, C, D, AB, AC, AD, BC, BD, CD, ABC, ABD, ACD, BCD, and ABCD. These can be generated fairly easily but given the highest level composite such as ABCD represented as a pair of parts such as ABC and BCD it is difficult to identify the other two possible next level down combinations (ABD and ACD).
2nd Oct 2011 To Do
- I'm not sure if the way I combine stimuli from independent senses is correct. I am using the same idea as for dependent sensors. Only novel (interesting) gets combined with novel and familiar (neutral) with familiar. However I do generate all 2^P-1 combinations.
- I need to combine two level-1 actons into a single level-2 acton when they are sequentially adjacent.
- I need to get the first reaction to a repeat to occur after the 3rd instance.
- I need to stop STM creation when a reaction is done.
- I need to apply generalization of the P-Habit trigger stimulus to all the next lower level down stimuli.
Looping
When a repeated stimulus occurs and I do a reflexive response, should I be expecting a goal that is the same as the trigger with uninteresting concentration or should I have no expectation? If I get a repeat of the trigger after the response should I start creating a looping acton? If not in this situation when would I create a looping acton that is learnt to be effective in situations where a stimulus other than the trigger is being monitored for its change?
Creating Actons
If I go from B to C on sense #1 should I be building an acton structure that looks like this?
B -> Act C
/ \
Aa ; Ab
/ \
_ -> a a ->_
Which means I respond with an a and then given the kinesthetic a stimulus I orient. Should orienting responses be in my actons? [Read 8th Sept 2011] I had concluded that non-responsive orienting should not be responses in actons. But now as a result of 21st Sept 2011 the acton is identified using both the trigger and goal. Thus pairs in which the action is to orient are possible.
Combine this idea with the 2nd Aug 2011 and maybe all orienting is stored as actons all the time.
Also note that in the example acton structure when it is executed that B and _ are tested for at the same time. Does this address my problem with P-Habit formation? And if I combine two orients in this fashion does it address the reflex response after the 3rd repeat issue? This issue is all about expectation of the 3rd repeat and action habits are all about expectation.
Correcting Action Habits
For updating purposes an action habit must be identified as a trigger and goal pair and the most recent action that gets you from one to the other replaces the last action. [Read 21st Sept 2011]. But for finding the most interesting thing worth doing in a given trigger situation we must go through the list of actions for the given trigger looking for the most interesting goal. But if we come across an older one in the list with the same action as a more recent one but with a different goal we should ignore it. Or should this one have been removed when more recently we found out that the same action produced a different goal?
3rd Oct 2011 Identifying an A-Habit
I’ve really got to get this correct. The idea that an action habit is identified by a trigger - goal pair allows there to exist simultaneously multiple instances of the same action for a single trigger each getting a different goal. There are old ones and more recent ones. This does not make sense. We want to know which goal will be achieved given the trigger and action. If the same trigger and action get multiple goals then we need to find the common element in the goals that the action always gets. This is because we should only have one entry for a trigger and an action. It should be based on the most recent experience. We could have the same goal appearing many times for the same trigger each with a different action. This means there are many ways to achieve the goal from the given trigger.
We also want to use and reuse just the trigger and action in the subconscious execution of action habits. They will point to trigger – action pairs and once in place should not be removed. The goal can be changed and refined but not the action.
So what are the rules for updating the acton structure given a goal that is not met? If there is a common part of the goal between the expected goal and actual goal one would expect this to become the new goal for the given action habit. See the 2nd test case run of 2nd Oct 2011. This will tend to reduce goals to just the kinaesthetic part of the input stimuli for primitive actions because these stimuli will be produced consistently for the given primitive actons. The test run does bring up the difficulty that the attention is attracted to the unexpected – missing stimulus. But this is true only if it is more interesting than the action habit being done.
What we want in test case #5 is for the action habits to not include the non-kinaesthetic stimuli in the goals because it keeps on changing. But in test case #6 we want the non-kinaesthetic stimuli to be part of the goal because it is determined by the response.
If for a given trigger there are multiple ways (actions) of getting the same goal then we tend to stick with the most recent way even though it may not necessarily be the most efficient. The older ways are still available should the more recent one fail on us.
S-Habits as A-Habits
However if we make the orient / do nothing responses into action habits given one trigger and do nothing there are many possible goals. Given the trigger – action combination for identifying actons the goal of the trigger – do nothing acton would be changed to its new goal whenever new sequences are experienced.
A do nothing – orient response actually says pay attention to the where part of the goal stimulus when being done consciously. If it is being done subconsciously the goal is not used so how can the pay attention task be done? Does it take the form of the trigger in the Aa part and an orient action must always be placed in the Aa part? Thus A,B would be obtained by the sequence A->_ and then B-> _. How would A,B,C be uniquely represented?
A -> Act
/ \
Aa ; Ab
/ \
A -> Act C ->_
/ \
Aa ; Ab
/ \
A ->_ B ->_
Which is not symmetric – The B,C might form first. What about?
A -> Act
/ \
Aa ; Ab
/ \
A -> Act A -> Act
/ \ / \
Aa ; Ab Aa ; Ab
/ \ / \
A ->_ B ->_ B ->_ C ->_
And I’m doing all this so that the repeating A can be reacted to after the 3rd A which is doing the AA sequence expecting an A which it gets and is boring. And this is because performing action habits sets up expectations. What if expectations were set up by STM instead for orienting? This would mean forming the A,A binon so it can be given an interest rating. Or a special flag on every binon to indicate it has repeated only once so far. On second repeat it is boring. Or maybe it can’t change from neutral – familiar to uninteresting in the same cycle? It has to repeat a second time.
4th Oct 2011 S-Habits in A-Habits
On further thought it does not make sense to use A-Habits to represent S-Habits and the do nothing/orient response. STMs do such a good job of creating and recognizing them and the triggers in actons can be S-Habits which are being waited for before the action part gets done. This reverts my thinking to the 8th Sept which is consistent with my latest thinking about identifying actons.
6th Oct 2011 7:30am Acton Identification
The conscious part of the acton is all about the goal – the final state. It does not say, given this trigger this is what to do to get your goal. It says, no matter what your trigger state is, this is the acton to get to your goal. So all the actons that get the same goal (get to the kitchen) are all grouped under one acton based on the goal, not on where you are / what state you are currently in.
This is a major conceptual shift. Now actons are combined under a goal that is used to identify them. So for goal stimulus B the 1st acton that got to B as a result of response x was T1 -> x. The next acton that gets to goal stimulus B as a result of response y is T2 -> y. These two get combined into a single acton so they can be enabled in parallel when goal B is desired. If we have T1 or T2 as the current stimulus the appropriate one will fire. What if we can also achieve the goal stimulus B when we respond with z after T3? We now have T3 -> z to add to the list of actons that could be enabled in parallel. Effectively we have three ways to get to B, one from T1, one from T2 and the third from T3. We can’t combine the 1st two into a binary tree. What would be the trigger required to enable these two. We could use a “don’t care” trigger that does not need recognizing before enabling the two.
And when we have two senses, one of which is kinaesthetic, and the stimulus B^_ repeats we respond with a. If the next stimulus is C^a we could form the following learnt actons. For goal C we could form the two B->a and _->a. For goal a we could form the same two Aa = B->a and Ab = _->a.
Once again the question arises. Should the orient response be incorporated into these new actons such that to reach goal G when we are at T we don’t need to do anything? We need to consider how the sequence B,C,D would be represented. Let us assume that the response b gets us from B to C and response c gets us from C to D. We end up with the acton for goal D – C->c and the acton for goal C – B->b. We need to create from these the one that also says for goal D you can use B->m where m is the combo of Aa = B->b and Ab = C->c. They will end up being done sequentially since Aa will result in stimulus C occurring. Instead of building the binary tree for this we could just add B->b to the goal D list and have it enabled with all the others. The whole idea behind the tree was to have a stimulus enable an action habit that used other trigger stimuli to perform the action. However there is also the idea of when does the acton stop / finish. It is not in the business of checking for the goal. So Ab’s are always actons that if done get the goal. Aa’s are always restarted for looping and they produce stimuli for triggering Ab’s. Thus when T1->x is combined with T2->y and T3->z each must take up an Ab position since they all reach the final goal. They would have to have empty Aa’s that act the same as the idea of “don’t care” triggers.
Another approach is to have a list of just the final actons (trigger and response) that achieve the final goal in the goal list. Then if we want to achieve the final goal we enable all these. Then for all their triggers we treat them as goals and enable all the actons for them. And do this again for their triggers until all preceding actons are enabled. As preceding actons match their triggers and perform their responses they continue to be re-enabled in case they repeat or their situation arises again during the pursuit of the final goal. Once one of the actons in the final goal list matches its trigger and does its response the whole structure is disabled. [I can see I’m possibly designing the cerebellum from basic principles at this point]
How does the structure get disabled because something else is started instead? Could we have only one master goal structure active at a time? How does it get disabled because none of the next triggers are found? Does it have to have matched at least one enabled trigger for it to continue? How are parts of it changed, deleted or added through learning?
15th Oct 2011 Acton Execution
I've been implementing the acton structure as described in the second to last paragraph of 6th Oct 2011. All final stage actons are marked active and preceding ones are marked repeater and remain active in parallel until a final stage active one occurs. Then the entire structure is disabled. I've now started to address how it is modified as a result of learning. I am also thinking about whether I need it to contain no action / orient responses. When we are holding our head erect we are dynamically monitoring the muscle positions / tensions and making sure any deviation from erect caused by non-intentional forces are compensated for by sending appropriate responses to the neck muscles. If the position is as desired then we continue to send no response. This would seem to imply the no response with a trigger and goal that are the same is being done to maintain the position. Given multiple devices / muscles this must be done in parallel for each muscle. If while this is happening we are moving a hand muscle to a new position based on a sequence of learnt actons and it fails, how do we correct it? First the failure is not recognized by the sub-mind executing the actons. It has recognized a trigger stimulus and produced the learnt act / response. The failure must be recognized by the conscious mind because it has recognized an unexpected S-Habit. This means that all S-habits that result from actions being done (including the orient / no response action) must be known and expected so that the unexpected ones can be recognized. What happens to the active A-Habit actons? They must carry on because we still want to hold our head erect. And we want to add any newly tried responses to get our hand to the desired position to the active set of actons.
If I want to pick up the spoon with my right hand and I am sitting at the table with the spoon visible I will not activate the get out of bed acton to go to the table. Thus as the proceeding actons are being activated to reach a goal should one of their triggers be recognized the activation process will stop and the associated acton will be done.
22nd Oct 2011 Parallel Sequences
If two or more independent but familiar sequences occur in parallel, one on one sense and the other on another sense, why aren't all sorts of novel P-Habits created? The two sequences can be out of synch at many different positions. Thus many possible novel P-habit combos will be created and attract attention. But when this happens to us, we do not recognize them. I can hear any number of words being spoken while looking around a room and never combine the two streams of information. But if I am watching the lips of the person speaking I do form the P-Habits and recognize the sequence of them. Each individual sense sequence is familiar, as is the combo sequence. However we do not succeed in reading lips very well.
The solution is based on recognizing the familiar combination of P-Habit parts first and only then combining the unfamiliar parts. Thus the parts that have proven dependent before are combined first. [24th Aug 2008]
23rd Oct 2011 Actons, sequences and trees
Actons do not recognize sequences of stimuli. That is, their triggers are never sequences. They are always given a simultaneous instantaneous stimulus and the action to perform. The A- Habits are linked to simultaneous instantaneous goal stimuli. The backward chaining that takes place when activation takes place sets up their sequential nature. So if sequences are not triggers or goals for actons then where do they come in? They are recognized consciously for repetition purposes and novel ones are recognized too. The alternative is there is some form of tree structure to actons and their activation. My first good acton structure design used a binary tree structure. But when I started backward activation from the goal to the current trigger I flattened it to the lowest primitive response level. So what is the advantage of a tree of actons and how would it involve sequences? We are conscious of the idea of goals and sub-goals but that does not necessarily mean the subconscious action habits need be structured in this fashion.
While we subconsciously execute the action habit our conscious mind is recognizing the sequence as a tree and is finding it familiar. Thus the conscious mind can think of things or be distracted by other more interesting stimuli while we execute an action. The conscious mind builds up sequences that actions are producing. But when you learn a new action as a result of not getting an expected goal (getting something else instead) is it at the most primitive response level that the action gets corrected?
25th Oct 2011 Novel Combinations
So I need to try out the situation where only the level-1 S-habits are triggers and goals for action habits. But I would still be forming the S-Habits for conscious recognition. I also want to try out not combining two novel S-level 1 stimuli into a novel S-Level 2 stimulus. I want to wait for the two level 1 parts to be familiar before combining. This seems a little anti-intuitive since to recognize an S-Level N sequence it has to be experienced N-1 times.
No I definitely need to form trees of action habits. When I do two actons (trigger and response) in sequence to get the goal (which is S-level 1) I need to form the conscious remembered sequence of trigger, acton (the pair) and goal. Then I can consciously relate trigger to goal even though a sequence of two actons was done to get there. When I want the goal the acton tree will be activated backwards as already described.
Many orient goals
An orient response for a given trigger may have many goals (next stimuli). Is this a problem? It is a problem only if the goal (next stimulus) is interesting, that is it would be happening consciously. Once one has been learnt it should not be started / activated so how many possible next stimuli could occur should not matter. [An unfinished thought] Given the interest in a next stimulus all …?
Parallel Actions
As I was walking to the bathroom I noticed a drawer not quite closed. As I went by it I consciously started an action habit in parallel to close the drawer. I thought about the goal of a closed drawer and did the series of actions to close it as I walked by. It did not interfere with my walking to the bathroom, which remained my goal. So I started a conscious action habit while I was executing a totally different action habit subconsciously. The two used different resources and thus did not interfere with each other. Did I have two goals conscious at the same time? Both goals were decided on by thinking about them. Going to the bathroom was going to take longer to do but the interest in the two goals was equal. So when thinking about goals we start action habits in parallel. I didn’t lose the goal of the bathroom because the subconscious action sequence was already active. After the close drawer was done I was still doing the go to bathroom. I may have observed my actions and remembered why I was doing them. If the goal was biologically motivated then this motivation would still be there to give me reason to go to the bathroom. However, if it was to get something that I had thought about that I needed the thought must have still been present in my short-term memory or something in the environment (which includes my body) must have reminded me of my goal. If my goal of going to the bathroom was to get some scissors then when I got to the bathroom it may have been the sight of the room that reminded me of the thought about goal. When I originally thought about the need for scissors I also remembered the image of the location in which they could be found, the bathroom.
Is this how we learn to join two actions in sequence? You think about the goal and then think about a preceding trigger stimulus as a sub-goal. You know how to get the sub-goal from where you are currently (i.e. have a learnt action habit). You know how to get to the goal from the sub-goal state and thus you do the two actions. If successful they are joined. What part of this process needs to be conscious so it is learnt and can be then used as a known action habit?
26th Oct 2011 Binary Tree
Yesterday I described the anti-intuitive nature of only combining two sequential parts when they were familiar and not if either was novel. It is possible that we as humans can form much higher level combinations of novel parts on just one sequential experience because we do not have binary neurons. With multiple inputs into a neuron it can wait for many sequential stimuli before firing and thus we do not need to re-experience the sequence again and again before the whole is recognizable.
27th Oct 2011 Action patterns
The following action learning patterns take place:
- Reflexive – Have a trigger, try a response and don’t know the goal. Get a goal stimulus.
- Repeat of a new action habit – have a trigger, do the response we have, know and want the goal so perform at an interesting concentration level.
- Think based on a trigger and do the response of an action habit expecting goal with expected interest at this thinking level of concentration. Another thought about action habit can be done in parallel.
- A partial match of the trigger results in trying its response.
- Consciousness detects a goal not reached by using the sequences created that it expects from STM processing. Action habits don’t check goals they just do their actions if they are active and detect their triggers.
- A thought about sequence T A1 Sub-Goal A2 Goal gets started if the goal is interesting. If T A1 Sub-goal is successful and reached then we will start the Sub-goal A2 Goal habit and if it succeeds we form the A3 which identifies T->A1 and Sub-Goal->A2. And we keep T A3 G for future use.
Device feedback
Consider a single muscle with a single kinaesthetic sensor providing feedback. We want a kinaesthetic goal stimulus based on the current kinaesthetic stimulus (state). We know the sequence of responses to achieve the goal and what to do if we get any unusual kinaesthetic stimuli on the way to the goal. I want to hold my head erect. There must be an always active habit maintaining this while I’m awake. So every device has an active habit maintaining its current position / state. This means we must always have a set of independent kinaesthetic stimulus goals that are being maintained. A new goal is another / different combination / configuration of these independent kinaesthetic stimuli – a new set point. Each device has learnt the response sequence to get there from here for all possible current states. The kinaesthetic sensor must continue to give / read the goal value / reading while the device is at the goal, in that position. Do you send changes to the device or a set point goal? Or do the actons send the changes necessary to reach the set point based on the current reading? For muscles the changes necessary are based on the difference between the current and goal readings as in a PID controller. But if these readings are symbolic then differences can’t be calculated from the readings. Differences must then be the pairs of symbolic readings comprised of the trigger and goal. The conclusion is that the responses / signals sent to a device are not the same as the values obtained from the kinaesthetic sensor on that device. The signals sent are change amounts, the readings returned are the current positions. The signals sent are not returned as the sensor readings as I have been trying to do.
2nd Nov 2011 Binary Tree
Back to the 26th Oct idea. It feels as though when we recognize a long sequence of novel stimuli we get bored on the 3rd time that it repeats. It is as though we have formed the highest-level tree structure representing the entire sequence after just the first sequence. But if we were practicing to create a long sequence, a new musical tune on a piano for example, we need to practice it many times before we have it all remembered. The tree structure chunking needs numerous repetitions to form into one complete chunk. This seems to imply that the learning of action habits only adds one more level of actons to the tree upon each time through.
3rd Nov 2011 A-Habit learning
If an action habit is active and an acton within it has just been performed but it fails how is it updated? How does success get used to create or modify actons? Obviously the acton fires if it is enabled / active and its trigger stimulus is recognized. But it does not monitor the goal. It just does its thing. This is the job of the conscious cortex recognizing patterns and sequences. So the recognition of the failure of an acton is consciously recognized by an unexpected (novel) sequence. Success is the conscious recognition of the expected. These two signals must be used when they occur to reinforce or change the acton that fired just before. So the enabled / active habit does not get changed. It is the executing one that gets removed or kept. It is removed by being replaced with the newly experienced one.
Muscles - devices
Muscles are devices that hold their position, even in the face of outside influences and change to a new position once given an amount of change to perform. Their position is the kinesthetic feedback sensor readings. The changes to perform are the responses sent to them.
4th Nov 2011 A-Habit learning
Is it possible to correct an action habit if it is not the one being concentrated on? If it was started at the interesting level and concentration is at this level then a success or failure can be associated with it. If it was started at the thinking or neutral levels will a distraction be a failure and update the action habit? Do we need to be doing it at the interesting level to update it?
Practice Mode
When an action habit is started at the thinking (expecting interest) level other thought about goals can cause additional action habits to be started in parallel. Do we have a mode in which we are practicing the habit and the success and failure signals are used and another in which the actions are just started and perform subconsciously without any chance of update? In the practice mode a goal is in mind and success or failure can be evaluated. Is practice mode done to validate an action habit and from then on it is assumed to be learnt and can be done subconsciously? [Read 18th Sept 2011]
So when does a learnt habit go back to practice mode? When a learnt action habit is being executed the first indication of failure is when we are conscious of an unexpected sequence. Putting the cereal box in the fridge generates such a sequence or instantaneous stimulus. [Read 21st Sept 2011] The unexpected stimulus causes us to stop and decide what to do. We then do it. This does not fix the action habit since it was not in practice mode. The next time the trigger situation arises we recognize it and remember the unexpected sequence because it was novel. Now our attention is attracted to this situation and we practice what we did last time. If it works in practice mode then it is learnt and can be done subconsciously.
How should I represent the novelty of a trigger-action-goal sequence that needs to be practiced?
- I can do as I was doing and use the novelty of the combo trigger-goal sequence and allow the orient response to be a response. This has a problem with the STM forming the novel multi S-level sequences on 1st pass but they are not created by the action habits. Can the action habits involving orient responses be formed at the same time at the multi S-Levels? Or
- I can assign a novelty / familiarity property to the acton which needs to become familiar through practice.
In strategy 1/ I also have to create multi-level action habits for the orient response if I am going to consistently react with a non-orienting response after the 3rd repeat of a novel sequence. Can I have STMs create these orienting action habits? Not necessarily, because some of them use a different sense for the trigger and goal stimulus. Maybe I should have a conscious STM for the attended to sequences of stimuli. But the fact that we must practice action habits many times to learn a long sequence of actions (chunking up smaller action sequences) seems to imply that 1/ would not work for recognizing long sequences. But strategy 2/ only reacts after the 3rd repeat of a novel sequence if the first reflexive response is the orient one. This still means that orienting is one of the possible responses used in action habits. This is certainly necessary to hold a device in the same position.
I believe I have to go with strategy 1/. Consider the 4th and subsequent repeats of the novel sequence. At what point do we turn on the action habit to do nothing and thus ignore the sequence?
5th Nov 2011 Sequence recognition
If an acton is active and waiting for its trigger stimulus that trigger could be an S-habit sequence stimulus. This would allow actons to not have to recognize sequences by performing orienting responses. This would mean the strategy is:
3. Orient responses are not used in actons and the acton needs a novelty / familiarity property to indicate practice is needed or not.
6th Nov 2011 Practicing recognition
Recognition of sequences must also be practiced. So in the logic for what to do next must be the test of the interest in doing nothing in order to get the same sequence again. Right now I only test for the interest in redoing an action habit. This would best be implemented by making the "do nothing" orient response one of the possible action habit responses. Or it could be added to the determination of redo-interest - to check for interesting sequences of stimuli in S-habits.
7th Nov 2011 Sequence Recognition
I need to be able to recognize what will happen next whether or not there is an action done. This is because I need to be able to recognize anything that is unusual / unexpected. This is not the same as recognizing / identifying a sequence as a chunk / S-Habit. This would mean that for every stimulus I need a list of the "what could come next" stimuli. This recognition is taking place in STMs per sense but not between senses. Within an STM a new "what could come next" shows up as a novel stimulus or novel sequence.
Pay attention acts
Orienting responses of do nothing still imply an act. That is what to pay attention to. The where part of the goal stimulus.
Conscious sequences
I believe that the sequences that STMs are creating remain subconscious until they are recognized as a whole. These wholes are the ones on which we do the "what comes next" processing. The wholes are either a sequence of novel stimuli which is terminated by a familiar one or a sequence of familiar ones that is terminated when a novel combination of them occurs, a novel stimulus occurs or a repeating sequence happens.
8th Nov 2011 Conscious sequences
I've been trying to implement this. Sequences of novel stimuli form novel wholes that become conscious when followed by a familiar stimulus. If the familiar stimulus is not conscious because it is the 1st part of a familiar whole sequence that has become conscious the preceding novel whole can be attended to. If the familiar stimulus is conscious because it is an S-Level = 1 whole sequence then I have the novel whole and the familiar stimulus to deal with in a single attention cycle. Maybe a novel sequence that becomes conscious does not have a "what comes next" or a reflexive orient response. It just gets created and might be a goal but is not a trigger.
The other question is when STM has combined several familiar but subconscious stimuli into a conscious whole sequence does it continue adding more familiar stimuli to the sequence or does it start a new combination of familiar stimuli to form the next conscious whole?
10th Nov 2011 STM Processing
What do all the following sounds have in common? Tinkle, Twang, Thump, Thud, Clank, Clang, Clink, Click, Crackle, Clap, Snap, Whip, Rattle, Rustle, Ring, Ding, Wham, Splash, Fizz, Sizzle, Hiss, Zap, Zing, Rumble, Mumble, Pop, Plop, Groan, Murmur, Gurgle, Quack, Baa and Oink. Other than the fact that they are all onomatopoeia words they are all sequential sounds that we do not consciously subdivide into their parts. These are what STMs are designed to recognize. Once they become conscious they are treated symbolically and can be triggers or goals for action habits.
Permanent processing
Once a conscious stimulus has been explored completely (all responses have been tried and no more actions produce goals of interest) it becomes permanent. It is now no longer treated as a trigger. Its only response is to orient. It is combined sequentially with other conscious stimuli to form stimuli that can be treated as triggers. This is what we do with words to form phrases. The second stimulus with which a permanent stimulus is combined need not be permanent. But can a permanent stimulus be a goal. It was most likely a goal for some action habits before it became permanent. What happens to these action habits? And what happens to the action habits for which it was a trigger?
Stimuli states
I believe I need a stimulus state called Familiarity that has values of Novel, Familiar, Conscious and Permanent. Then within the Familiar state the stimulus can have an Interest level of Interesting, Neutral and Uninteresting (possibly also expected interesting and expected uninteresting). In the Novel and Familiar states the STM per sense continues to combine the stimuli into longer sequences.
12th Nov 2011 Conscious STM
I now am starting to implement conscious STM. It will be used to detect any repetitions of sequences of conscious stimuli. It will also be forming sequences of conscious stimuli across actions that are between them. The conscious objects put in this STM would include any P-Habits that have been recognized. Think of a P-Habit as a certain configuration of sense/sensor values. At the conscious level it could be a sequence of P-Habit configurations that is the object of attention. [Read 16th Aug 2010]
13th Nov 2011 Length of Conscious STM
Things are removed from Conscious STM when they have served their purpose. They have either detected a repetition in the attended to stimuli or they have noticed something unusual. Something that was unusual was unexpected and forms a novel sequence.
Conscious STM generation
Should I be generating sequences in Conscious STM ahead of what the action habits are creating? Even if the action habits include the orient response the conscious STM generates more. For example if A, B and C are all conscious and A, B and C occur in order. A->_B creates A,B. B->_C creates B, C but Conscious STM also creates (A,B),(B,C) as well. And the 2 action habits have yet to be combined equivalent to this 3rd level sequence.
14th Nov 2011 Conscious STM
Currently the sequences generated by Conscious STM are not necessarily the same as those from the senses. They may be sequences involving stimuli from two or more senses. But they still need to attract attention. However they only get generated after the attended to stimulus has been used as a Goal Object. This is before the attended to stimulus gets used as a trigger.
At the same time I should be creating Actons corresponding to what is being generated in Conscious STM. Thus I have concluded that sequences of conscious stimuli don't grow larger than the action habits that are needed to pay attention to them.
I have decided and I have working the idea that orienting responses are action habits. These handle the situations where the conscious stimuli form sequences that are familiar but are not caused by one’s own actions.
Acton structure
On 23rd Oct 2011 I questioned the flat versus tree structure for action habits. I have decided the tree structure is necessary such that given a trigger and a desired goal there is only one acton between them. This acton is the root of the binary tree. An acton is activated / enabled when the trigger is conscious and the goal is wanted. Each Acton has a trigger that must be matched to perform it. When the acton is performed its two parts / slaves are enabled. If their triggers match with the current stimuli (conscious or sub-conscious) they are performed. The two slaves could then be active in parallel. They could also be active in sequence if the trigger for the second needs the 1st slave to be performed. The two slaves are being performed sub-consciously, dependent only on their triggers being available. As a result of learning and practice the acton tree for "go to kitchen" would contain slave actons with triggers such as "in hallway", "sitting in office chair" and many other starting 'states'. It might not have one with the trigger being "lying in bed". This would require some conscious thinking / problem solving to be done to join together the "get out of bed" action habit and the existing "go to kitchen" action habit. Once practiced the "lying in bed" trigger will be a new slave trigger.
Disabling Action Habits
The big unknown is how do the enabled actons get disabled? Back in 31st July 2011 I was assuming that the acton execution tested the goal stimulus. But I now know this is not the case. Goal stimuli are conscious stimuli and recognized when practicing action habits. But when action habits are being done sub-consciously there are no goal tests being done. It is driven purely by trigger recognition. Should I maintain a list of thought about goals and the actons that were started such that when the goal is reached the acton gets disabled top down? If so what happens when the goal is not reached (the cereal ends up in the fridge), we are distracted and seek some other unrelated goal or it's forgotten. We end up with enabled actons. Another approach would be to say that every lowest level acton contains a complete set of all responses that must be sent to all devices. It is thus a configuration response. An action habit would then contain a tree of such actons. Should any alternate higher level acton become enabled all the currently enabled actons would be replaced. I don't immediately see how this might work. What really needs to be done is the disabling of the actons such that they are not waiting for their triggers. The sub-actons may have been enabled but never done. For example the "open kitchen door" may be part of the "go to kitchen" action habit but never triggered because the door was already open. How does a new goal being sought replace or disable currently active action habits? There are only three handles on the current action habits. These are;
- the highest level acton with its goal,
- the set of devices that are to be involved if the triggers happen and
- the known stimulus sequences that could happen during the performance of the action habit.
Three is the identifier for the trigger - acton - goal and therefore contains 1/. Number two does not seem feasible because there may be devices involved in a currently enabled acton that are not in the new action habit. Somehow the answer lies with the sequence of stimuli that are expected with reaching the goal. There must be a conscious decision to 'drop' the current action habit when starting another one that is not trying to be done in parallel. The possibilities are 'do in parallel', 'do instead', 'do after' and 'do next'. 'Do instead' interrupts the highest-level goal action habit and drops any enabled action habits. 'Do in parallel' and 'do after' get combined with the current one. 'Do next' is done when the active one reaches its goal. 'Do next' is possibly a 'do instead' when the goal has been reached.
Recognizing Action failure while thinking
Recognition of trigger stimuli is being done by sub-consciously executing action habits. But this cannot attract attention when the sequence of trigger stimuli fails. So when one is thinking while the action habit is being done one is not consciously paying attention to the trigger stimuli. One is not practicing the action habit. But one's thinking still can be interrupted by a failure to recognize the trigger sequence. Is this happening because there is always an S-Habit being done in parallel to recognize the known stimulus sequences? Thus novelty in the sequence would be more interesting than thinking. This would imply that the conscious STM continues to parse conscious stimuli while we are thinking. In this case the name of 'conscious stimuli' is a bit of a misnomer. They might better be called explorable stimuli.
15th Nov 2011 Acton Structure
Conscious STM continues to subconsciously parse known explorable stimuli while one is thinking. Actons don't disable themselves. The acton structure is therefore quite simple. Two slave actons are enabled in parallel. They may execute in parallel or in series. They both stay active until disabled from above. The current highest level acton and its goal are the only things being done. Any other acton started at the same time must be added in parallel or done instead.
No, this is not going to work. I need an acton that has two slave actons, both of which are active but only one can be done. They are mutually exclusive because they use the same resources / devices for different reasons. When the 1st is executing the second must not and vice versa. For example walk and step over obstacle. They are both enabled waiting for their trigger conditions. When one executes it must inhibit the other. When the one completes it must remain active ready to do it again and stop inhibiting the other. I will call this acton with two mutually exclusive slave actons an Exclusive Acton (E-Acton).
I also need an acton that has two slave actons, both of which are active and both can be done in parallel because they do not share any common resources. They can start executing independent of each other out of synch. I will call this acton with two independent slave actons an Inclusive or Independent Acton (I-Acton). This would allow one to do something with ones left hand while ones right hand does something else. Could this acton also be used to do two things in sequence since the trigger for the second is only set-up when the first is complete?
I also need an acton that will repeat. It will re-enable its two slaves once they are both complete. I will call this acton a Repeating Acton (R-Acton). These descriptions include the concept of complete or finished executing. This would be when an acton is enabled, its trigger has occurred and it has started its slaves. And when its slaves are complete it is complete. I need to re-instigate the idea that the 1st is always a repeatable slave and the second has to complete for the master to complete. If an I-Acton is to remain active for the duration of the action it will need to be a first slave. If an I-Acton were to be used as a sequential one then it would be a second slave. An E-Acton must always complete when either of its slaves completes. It must be a first slave so that it gets reactivated.
18th Nov 2011 Practice S-Habits
I have to make sure the conscious STM is not creating long sequences of S-habits that have not been practiced and learnt. Otherwise it is creating much longer sequences than learnt and is not being used for its primary purpose which is to detect any unexpected stimuli that an action habit has caused. This includes unexpected stimuli from doing nothing since the orient response is being kept as action habits. Thus when the conscious STM creates a combination stimulus and that combination stimulus has no list of actons then the combination should be removed from the STM, leaving just the most recent conscious stimulus.
Permanent Stimuli
Permanent stimuli are stimuli that are not being explored any more. They get combined with the next stimulus that occurs and the combination is explorable. But do permanent stimuli get combined with the next conscious stimuli no matter what sense or do they only get combined with the same sense stimuli? The software attention algorithm does not lend itself easily to combining permanent stimuli with next any sense stimuli. This is because it is only conscious stimuli that attract attention. Permanent stimuli are not conscious anymore. So I've concluded that permanent stimuli are processed by the per-sense STMs and only get combined with same sense stimuli. Effectively they revert back to the familiar state that stimuli are in before they become conscious.
30th Nov 2011 Combining Permanent Stimuli
A key question is; with what other stimuli types can permanent ones be combined? Obviously it can't be the 2nd of two in which the first is conscious because we want to react to the first. It could be the 1st of two in which the second is conscious. The resulting combination would start out novel and then go through familiar until it becomes conscious.
The 10th Jan 2004 remark is good. Permanent stimuli only occur in a dynamic environment because the stimulus that follows a permanent stimulus must be different than the permanent one without any response being made. A good example is listening to someone talk.
The 10th March 2006 remark mentions that a permanent stimulus followed by a novel one does not get combined with the following novel one.
The 16th March 2006 suggests only combining permanent with permanent.
I don't believe permanent stimuli should be combined with novel or familiar ones. It should act as a delimiter such that a series of familiar or novel ones end when they encounter a permanent one just as when they encounter a conscious one.
We still have to form sequences that include permanent stimuli. The permanent stimulus must act as an indivisible unit.
1st Dec 2011 Combining Permanent Stimuli
Novel and familiar stimuli are like phonemes. They are not conscious and never are. Syllables are combinations of phonemes. They can become conscious at a young age such as when we learn to repeat ba ba ba or ma ma ma. Eventually syllables become permanent and get combined to form words. We do not combine syllables with phonemes to form new syllables. As words become permanent we combine them into phrases and sentences. It is usually at this level as adults that we are conscious and continue to react to what is said. This leads me to the conclusion that permanent stimuli only get combined with other permanent stimuli and the results are newly conscious stimuli.
But when trying to implement this, two important questions must be answered. Do the two permanent stimuli need to be experienced consciously one after the other before the combination is formed? And is the combination restricted such that the two permanent stimuli must be from the same sense? The answer to the second question would seem to be yes. This would imply they are formed by the STMs dedicated to each sense. It would also seem to mean that they might not need to be attended to before combining, similar to familiar subconscious stimuli. However can a permanent stimulus continue to be attended to consciously? Certainly not for answering "what to do next?" But maybe it is attended to as a goal stimulus. This would also imply that it has a possible interest level. But I am assuming they are of neutral interest and never change. So I think I will try to model them as neutral interest and never conscious. They are very similar to familiar stimuli. They can't be combined with conscious stimuli and they are discarded by STM if followed by a novel or conscious stimulus. If followed by a familiar stimulus then we wait to see if the sequence of familiar stimuli ends up being conscious or permanent.
4th Dec 2011 Permanent Stimuli
Words like 'boat', 'tree' and 'hammer' have become permanent in our minds. We do not normally respond to them, we wait for the rest of the phrase or sentence. Sequential sounds like 'gwong' and 'friba' are novel. When we hear these combined in a sentence they stand out as novel, we become conscious of them and pay attention to them. In the sentence 'The gwong tree fell over.' we have a sequence of permanent (The), novel (gwong) and permanent (tree fell over.) parts. The novel part made the preceding permanent part conscious but we might not have paid attention to it. The novel part came to an end when the first permanent part of the end part was recognized. The sequence 'gwongt' and 'gwongtr' were novel until the word 'tree' was recognized. Then 'gwong' became conscious and interesting and attracted attention. This seems to imply that the permanent stimuli are promoted for conscious processing however they are not reacted to.
What about the sentence 'The boat tree fell over.'? Two permanent stimuli have been combined to form a novel sequence that attracts attention. It can even sound like 'botree'. Neither of the two parts attracts attention like 'gwong' did.
I therefore think that permanent stimuli are always passed on to conscious processing like conscious stimuli. And permanent stimuli can only be combined with other permanent stimuli from the same sense. Also the permanent stimuli are then the smallest stimuli for combining in conscious STM.
10th Dec 2011 STM by Sense
STM seems to be the best and longest for hearing. We parse speech in the temporal dimension. We don't seem to have a very extensive STM for sight. The complexity is in the recognition of multiple objects that make up a scene. STM is used though for moving objects. The sense of touch is also in the recognition of the source rather than the sequence. Taste and smell have very little STM. For kinesthetic stimuli the sequence is very important for coordinating the actions. But kinesthetic stimuli are invariably caused by actions. Thus the resolution of the action primitives is at the same resolution as the raw stimuli. Thus STM on kinesthetic stimuli is forming sequences of stimuli caused by action. Whereas for sound the environment is dynamic and STM is recognizing sequences where no action has been involved. But when action does cause sound the sound pattern produced must be structured using binons just as kinesthetic stimuli are structured in response to actions. That is to say the structure of the STM sequential patterns should be independent of the resolution of the actions that may or may not have caused them. This corresponds to the idea that the cortex continuously forms the sequential patterns on all senses independent of what level of detail / complexity at which they become conscious and subject to action / active exploration. This begs the question: when do the STM sequential patterns drop out of STM? Obviously this is not just when they become conscious as I have been currently modelling. They certainly drop out when there is a repeat in STM. But if we are listening to a piece of music that does not repeat and is quite dynamic and independent of our own actions how long does STM last before it fails to recognize a repeat because the previous sequential pattern is no longer in STM?
The implication for permanent stimuli is that they are still conscious stimuli but there is no reaction to them. And STM continues to combine them and their parts in a binary tree. The criteria for a stimulus remain the same. The familiarity changes. Both conscious and permanent stimuli are interpreted as familiar for sequential pattern formation.
12th Dec 2011 Conscious processing
There are stimuli that are formed before and after conscious processing. The current per sense 'before' STMs create pre-conscious stimuli. Trigger and goal combos of conscious stimuli are formed to detect repeats and to have a place to keep the interest level of the trigger goal action habit. These stimuli become permanent and can be combined into longer conscious patterns. Permanent stimuli should be combinable with permanent and conscious stimuli on the same sense. The sub-conscious parts of conscious stimuli may become conscious if they exist independently of their following stimuli. What I need is a per-sense 'after' conscious STMs.
17th Dec 2011 Sequence Recognition
I'm still trying to sort out the rules for processing stimuli sequentially. I now have realized that in a single sense before consciousness STM the sequences are formed up until a repeat occurs. The STM is not emptied until then. Then the repeated sequence becomes a conscious stimulus. They do not get removed when they change from familiar to novel or vice versa. We are always combining familiar with familiar and novel with novel at the same level. We never combine familiar with novel or vice versa at the same level. But this brings up the question what happens when a conscious stimulus occurs in this STM. It is obviously familiar but does the preceding unconscious familiar stimulus sequence get combined with it? Or does the preceding unconscious familiar stimulus sequence become conscious? The familiar stimuli that comprise the conscious stimulus occur sequentially and it is not until the end that the conscious stimulus is recognized. So the 1st familiar stimuli that comprise the conscious stimulus must be combined with the preceding familiar stimuli based on the STM composition process rules. What is taking place is learning to recognize sequences.
Can this sequential recognition learning be represented as action habits in which the action is "do nothing" but pay attention to the goal stimulus? (a) The 1st reflexive action to do for a conscious trigger is to "do nothing". (b) We only try a new action when the most recent recalled action habit goal is uninteresting and its expected trigger / goal combo is uninteresting. This must have occurred because of a past repeat of trigger and goal. (c) We continue to do the most recent recalled action habit if its goal is interesting or its trigger / goal combo is interesting. And (d) we do nothing if the most recent recalled action habit has a neutral goal and the trigger / goal combo is neutral. Rule (a) would work for a novel stimulus at any level of sequential complexity. Rule (b) allows us to wait for the 3rd repeat in a row before reacting. If novelty is used for interest in rule (c) then we continue to "do nothing" as novel stimuli sequences form and are combined. And if familiarity is neutral interest then rule (d) means "do nothing" continues while stimuli sequences form and are familiar. However the trigger and goal stimulus sequences would contain overlapping lowest level stimuli.
If this worked then there would be no need to identify the conscious sequences because they would just be ones that have actions that are not "do nothing". And recognition learning is just a subset of action learning. However for a STM to recognize any length of repeated stimuli they need to combine novel and familiar stimuli at all levels because a repeat can occur at any level. The rule to combine only stimuli with the same familiarity at any given level is a sequential version of what we do in parallel recognition to recognize an object. The parts of the object are all familiar, as are all the relationships between them at all levels of complexity. And this configuration or pattern remains constant for the object thus recognized. But the purpose of this object recognition is ultimately to identify the smallest unit of the environment to which attention needs to be paid and action associated. The advantage of parallel recognition is that all the stimuli are available simultaneously. A possible solution appears to be; treat all the lowest level sequential stimuli as familiar even when they are experienced for the 1st time. Then as the name implies sequences are being recognized. Also it seems reasonable to assume that a sensor that detects these stimuli at the lowest level has been built to recognize them thus they are never novel to it.
18th Dec 2011 Sequential Recognition
Yesterday's solution does not work at all levels of complexity because at some higher level we are still only combining stimuli based on familiarity. In order to recognize repetition we have to form all sequences at all levels independent of familiarity. Familiarity is not useful for this purpose and should not be a criterion for STM processing. It is useful for action decisions because novelty is pursued. So assuming STM creates all the possible sequences up until a repeat and all sequences are stored as action habits with "do nothing", will this work?
19th Dec 2011 Goal Recognition
Continuing my notes from 29th April 2011, what criteria are used to recognize a goal after a reflexive response is done?
- The next S-Level 1 stimulus
- The first novel stimulus that occurs after the response
- The longest novel stimulus that occurs after the response
- The 'whatever attracts attention next' stimulus
- A sequence as long as the trigger
- A conscious stimulus is encountered in the sequence
- A repeat of a stimulus is encountered in the sequence
Using the idea that the objective is to avoid getting a repeat of the trigger, a possible answer is you wait for up to half the length of the trigger stimulus sequence to see if the goal is going to be a repeat of it. If before this length of stimuli the goal is different from the trigger then the response has succeeded in causing a change and thus novelty. If the goal is the same as the trigger for this length of stimuli then the response has failed in causing a change. This fits nicely with the concept of similarity when using a binary tree for creating stimulus sequences. One needs only to recognize the first part of a two-part object to say it is similar enough for the purpose of recognizing the whole object.
And once the failure is recognized (half the repeat of the trigger) this goal is the trigger for another reflexive response. This time the length is a quarter of the 1st trigger and half this new trigger. Effectively a new response is tried faster and faster provided a failure occurs. But I have a problem with the two parts of a sequence that has many sub-parts in it. Using my binary tree structure the 1st part of a long trigger sequence is all its lowest level stimuli except the last one. I need to have a better definition of similarity. Using the current binary tree it would have to be recognized halfway down the tree.
And the next question is if something different from a repeat of the trigger is recognized then how long does one continue recognizing the sequence as part of the goal. The goal would seem to be everything that subsequently occurs as long as it remains entertaining or familiar. If a repeat occurred it would end the goal. If a stimulus sequence were recognized that was a trigger of an action habit with expected interest the goal would end.
Is it possible the stimulus that is recognized / attended to as the goal is not the one that comprises the next trigger? This may be true for action habits containing a do something response.
Attended to stimuli
The idea from 2nd May 2011 has some merit. It is that all the sequences generated on the STM are candidates for attention. This means not just the longest one generated. And the interest that attracts attention may not just be the interest in the stimulus that occurred. Attention may be attracted by stimuli that have associated interesting goals or interesting sequences (trigger-goal combinations). But don't these last two interests show up as expected interest in the trigger provided it has been propagated backwards to it from the goal or trigger-goal sequence. A repeat such as ABCDCD is one in which not the longest sequence attracts attention. STM effectively says the repeat attracts attention before we have finished creating the whole tree.
20th Dec 2011 Redo Interest
I currently create a combo object from the sequence of trigger and goal objects that comprise an acton so that I can store the novelty of this sequence and thus the interest in redoing it. However at S-Level = 1 and if the trigger and goal are from the same sense then the same sequence has already been created by the STM when the goal occurred. The creation of the combo for the acton then resets this sequence to familiar and neutral interest. I don't want this to happen. One solution is to not change the combo's familiarity and interest in this special case. Another solution is to add the novelty and interest property to the acton instead and not create the combo sequence stimulus.
22nd Dec 2011 Goal Recognition
When a reflexive response (a) is expecting a repeat of its trigger (B) (S-Level =1) with a desire to avoid it our attention is directed to the same sense and S-Level =1 stimulus (A). This stimulus (A) could be of neutral interest. At the same time the trigger goal sequence (B,A) may be novel and therefore interesting. Should this sequence be the goal achieved and learnt? Or should the goal learnt be the S-Level =1 stimulus (A). Does the response produce the sequence = the change? Do I want to get the final goal state (A) or do I want to get the change in state? When it comes to wanting to redo the habit will the A be wanted or the B,A sequence? If the goal A is wanted will the A-Habit be found? The sequence is interesting and is equivalent to the interest in redoing the A-Habit. I have goal interest and sequence redo interest to consider. Are goals always at S-Level =1? Does the reflexive response A-habit get completed / stored when one is distracted? Maybe yes. But it will then try to reproduce the same result and the chances are very slim the same distracting stimulus will repeat. If it does then maybe the response really did cause the distracting stimulus. But sequences are interesting as well as S-Level =1 stimuli.
Maybe the goal length cannot be longer than the trigger. Or maybe I should treat it as a pure state machine. Nested states would correspond to P-Habits. But what do I do with state transitions that are orienting ones? This might be the better approach and then see what other things have to be modified.