Research - Specifications
Given that Adaptron must function in any configuration of robotic senses and action devices it is necessary to clearly and precisely specify these senses and devices. It is also necessary to have a practical specification for learning and thinking. These do not always correspond to current cognitive science theory. Adaptron’s architecture and design are based on these specifications. These specifications are not complete and are intentionally simplified for highlighting only those aspects that are important for the software research. Key terms are underlined when defined. These specifications will be updated at the same time as the architecture, design and research notes.
Senses, Sensors and Stimuli
Senses are the devices that measure the environment and the state of the body. The five most commonly described human senses are the ones that measure the environment. These are the external senses. They are:
| Body part | Sense of | Properties or types of information measured |
|---|---|---|
| Eyes | Sight (visual) | Light - colourful images, distance |
| Ears | Hearing (oral) | Sound - noises, frequency (pitch), direction |
| Skin | Touch (haptic) | Contact - pressure, temperature |
| Tongue | Taste | Chemicals in solids and liquids - flavours |
| Nose | Smell | Chemicals in the air - aromas |
Additional internal senses gather information about the state of a human body.
These senses include the following:
| Body part | Sense of | Properties or types of information measured |
|---|---|---|
| Inner Ear | Balance | Orientation, acceleration, roll, pitch and yaw |
| Joints/Muscles | Kinaesthesia | Position, strain / tension |
| Stomach | Hunger | Building material and fuel |
| Hypothalamus | Thirst | Hydration |
| ? | Time | Duration |
| ? | Tired, sick | Stored energy, state of healing |
There are other internal mechanisms that are similar to senses in that they are sources of information used by the brain. It may be that the nervous system generates or adds these properties to stimuli. These are often referred to as feelings.
| Sense of | Properties or types of information measured |
|---|---|
| Interest | Novelty, familiarity, boring, unexpectedness |
| Pleasure | Pleasant, neutral, unpleasant, painful |
There are a wide variety of possible non-biological senses that could be used in a robot. Some measure the same properties as animal senses while others measure properties that animals can not detect. They include, but are not limited to:
| Sensing Device | Properties or types of information measured |
|---|---|
| Video camera | Light, infrared (heat), Ultraviolet |
| Radio | Radio waves |
| Microphone | Sound, ultra-sound |
| Laser range finder | Distance |
| Thermometer | Temperature |
| Chemical detectors | Aromas, chemicals in liquids |
| Compass | Magnetic direction |
| Pressure sensors | Pressure, stress, tension, acceleration |
| Gyroscope | Rotation - roll, pitch and yaw |
| Anemometer | Wind speed |
| Radar dish / gun | Distance, Speed, and Altitude |
| GPS | Location |
| Tachometer | Angular velocity |
Each sense has numerous sensors that measure or detect the type of information. For example, a simple description of the ear: a series of sensors each of which detects the volume of sound at a particular frequency.
The most atomic simple stimulus or measurement is a single valued reading that has been detected by a single sensor from a particular sense at an instant in time. The sense determines the type of information, e.g. light, sound, pressure, tension etc. Sensors determine the source / location of the information, e.g. frequency (colour and pitch), the chemical, skin location etc. Sensors measure the value of the intensity, volume, or amplitude of the stimulus.
A stimulus may also consist of many simultaneous measurements and may also be a series of such stimuli over time. Simultaneous stimuli from multiple sensors of a single sense form a composite stimulus that we recognize as an object. Examples are the image of an apple that is detected by the retinal sensors and a chord on the piano that is made up of numerous simultaneous frequencies. The term object is used to mean a tangible (but not necessarily touchable) thing in the real world.
Stimuli that occur simultaneously on two or more senses are called parallel stimuli because the readings occur in parallel. An example is seeing a door move and hearing its hinges squeak at the same time. Another example is when you scratch your forehead you see, hear and feel it simultaneously. This is called multimodal perception.
A series of stimuli forms a stimulus that is called a sequential stimulus. An example on a single sense is the sound of a tuning fork that lasts for one or two seconds. Assuming there is only one pitch in the note this stimulus consists of a single frequency but its volume has a rapid increase when struck and a steady decay afterwards. Another more complex example is the sound of a car passing you on the street. Seeing a person's lips move and hearing the spoken words at the same time is an example of a sequential stimulus consisting of a series of parallel stimuli.
A general purpose model that can be used to represent most senses is a linear array of sensors each providing a measurement of the intensity of the sensed information. The sensors in an ear form such a linear array. Non-biological senses can also be mapped on to such an array. For example, a laser range finder provides distance measurements for particular angles. Each angle is a particular direction relative to the robot’s body and is equivalent to a sensor position. A more sophisticated model requires a two dimensional array of sensors and more than one sense organ to produce a three dimensional effect as in the sense of sight.
Most biological sensors provide stimulus values that are magnitude readings. That is, they have units of measurement and can be placed on a continuous ratio scale. But there is a resolution that separates one value from an adjacent one on this scale. Non-biological sensors may also produce stimulus values that are discrete symbolic readings. An example of a sensor that produces symbolic readings is a robot touch sensor that has been designed to detect the type of material with which it comes in contact. It could indicate whether the material of the object is air (touching nothing), solid (wood, metal, rubber, glass, cloth etc) or liquid (water, petrol, milk, mercury etc.). All the sensors of a sense provide either magnitude or symbolic readings. Thus each sense can be categorized as producing either symbolic or magnitude stimuli.
Another non-biological sense may have numerous sensors that have no relation to each other. Thus the sensors are independent and no concept of scale or relative position between them can be obtained. An example might be the sense that provides the physical configuration of a robot's body parts. There may be many motors in a robot each controlling the angle of the joints of its limbs or wheels. Each electric motor would have a sensor to measure its rotational position. The sensors would produce this position as a magnitude reading from one to 360 degrees. The combination of readings from these sensors is another example of a composite stimulus. So a sense may have discrete independent sensors as in the motor sensors or it may have dependent sensors in the form of a linear array as in our ear.
Note that senses (hearing, sight, touch etc.) are also independent of each other because they measure independent properties.
Table 1 contains numerous examples of possible senses that are based on the combinations of the two types of readings and two types of sensors.
| Magnitude readings / stimuli | Symbolic readings / stimuli | |
|---|---|---|
| Dependent sensors | Hearing, Laser range finder | Robot touch belt (described below) |
| Independent sensors | Robot motor positions | Farm animal recognizer (described below) |
A robot touch belt would consist of a linear band of touch sensors around its waist. Robot touch sensors were described above as able to detect wood, metal, water etc. A farm animal recognizer may consist of multiple sensors scattered at random throughout the fields and barns on a farm. Each sensor would consist of a video camera and a computer that is programmed to recognize and identify any animal that falls within its visual range. The symbolic stimulus values might be cow, pig, chicken, human, etc.
Note that a robot does not necessarily need to have its senses and action devices physically interconnected in a single body. The farm animal sensors could be part of a robot that controls farm animals by opening and closing gates and barn doors. Another example is a robot that controls the lights at an intersection. It could employ multiple video cameras and vehicle sensors under the road.
Dependent sensors of a sense may be circular or linear. In the case of hearing they start at a very low frequency and end at a very high frequency and are therefore linear. However in the case of a laser range finder each angle of direction corresponds to a sensor. With a resolution of one degree there would be 360 such sensors each measuring a distance. However sensor number one is adjacent to sensor number 360 and thus a circular array of sensors is required. The same circular or linear property is also used for describing the magnitude readings from a sensor. Volume readings are linear while the angle of a motor is circular.
Devices and Responses
For an animal or robot to move and have some effect on its environment it must use some form of output device that causes action. In most animals muscles are the bodily action devices. In robots the motion devices are most likely motors but can be other types of electrical devices. Other robot action devices would include speakers and lights. Responses are the output signals generated by the brain and sent to these devices to cause these actions.
The brain knows if an action has been done because it receives the kinaesthetic feedback stimuli that were produced by the sensors attached to the output devices. In the case of humans the feedback stimuli are from stretch sensors on the muscles and tendons. At the same time the brain is also getting feedback from joint sensors and other senses such as sight and balance.
It is important that all action devices have associated feedback sensors. This allows for the confirmation of the action as well as the detection of an external disturbance - one not produced as a result of a response. In a robot such feedback would indicate that a light has burnt out, a speaker is not working or a limb has been moved by an external force.
Consider a simple robot action device that allows it to move left, right, forward or backward one unit of distance at a time. The device has four commands (responses) it can be given. It would also require a feedback sensor that returns one of these four possible discrete / symbolic readings. This is called a discrete device because it is given discrete responses. A magnitude device is given magnitude responses, such as a change in rotational angle for a motor. Magnitude devices require magnitude feedback sensors.
Learning and Habits
There are three kinds of learning:
- learning to recognize,
- learning to do and
- learning to think.
Learning to recognize is also called pattern recognition or perception. Learning to do is learning behaviour and in this document is called action learning. Learning to think is the process of improving one’s ability to mentally model the world based on experiences.
All things that are learned are kept in memory in the form of habits. A habit is a recording of what stimuli were experienced and what responses were done (if any).
For pattern recognition there are two types of habits:
- parallel (spatial) and
- sequential (temporal).
Parallel habits (P-habits) record stimuli that happen simultaneously. Sequential habits (S-habits) record stimuli that happen in series. Action habits (A-habits) are used to record learned actions. Action habits are a sequential record of a trigger stimulus, a response performed and a goal stimulus.
Habits are like computer programs in that they can be performed. When an S-habit is performed it is triggered by the first stimulus in the habit and is expecting the next stimulus. When the next stimulus is perceived a sequence of two stimuli is recognized. A-habits are started when their trigger stimulus occurs. Then the response is produced and the goal stimulus is expected. Learning is the process of remembering these habits and performing them again in order to achieve a desired goal.
We perceive the environment as being made up of many objects. Each object is made up of different parts that are also objects. This forms a tree structure. S and P-Habits form tree structures that are used to remember the combinations of parts for later recognition purposes. Perception is the process of learning to recognize (identify) objects from stimuli. This is synonymous with the process of converting magnitude measurements into symbolic information that identifies an object. It also includes the process of learning to recognize complex objects composed of parts that are also objects. The composition of complex objects from less complex objects can occur at multiple levels in parallel and in sequence.
One of the goals of an animal or robot is to learn about its environment from the detailed stimuli that continuously flow from its senses. Once it has a memory of its environment the goal is to notice / pay attention to any interesting things that happen. Interesting stimuli are novel / unfamiliar. But to determine the novelty of a stimulus it must be “looked up” in memory. It is done through the subconscious performance of S and P-habits. Familiar stimuli match the existing recognition habits. New habits are created for novel stimuli and they may attract attention. Conscious stimuli are those to which attention has been paid. It is only in this conscious mode that action habits can be learned.
Attention has two modes in which it operates:
- attracted and
- directed.
In the attracted mode attention is attracted to the most interesting stimulus from the ones available as described above. In the directed mode we are concentrating on perceiving a specific expected goal stimulus. This mode is used for practicing and thus learning action habits. The concentration level is the level of interest in the action habit’s goal stimulus.
Action learning is the process of discovering and remembering the goal stimuli that result from the performance of actions / responses in given trigger situations. An action habit has been learned when it is being practiced and the goal is reached. However, if a stimulus occurs with an interest level higher than the concentration level then attention is attracted away and we are distracted from achieving the goal. If an unexpected but not distracting goal stimulus is obtained then we have learned a new action habit. Because it is important to learn an action habit when it is being practiced attention is focused on only one stimulus at a time.
Complex action habits can be made up of other action habits. There are three kinds of action habits:
- looping,
- sequential and
- parallel.
A looping action habit is one that is repeated continuously such as walking or clapping. Sequential action habits are ones that chain other action habits in series such as tying a shoe lace. Parallel action habits are ones that can perform other action habits simultaneously provided they do not make conflicting use of action devices. An example is speaking and walking. However speaking and eating is hard to do simultaneously because of the conflicting use of the tongue and mouth muscles.
Actions are performed in one of two modes:
- babbling and
- directed.
In the babbling mode actions are done at random from the ones that action devices can perform. In the directed mode we are concentrating on performing the action and perceiving a specific expected goal stimulus.
Learned action habits are selected for execution based on the occurrence and recognition of their trigger stimuli and the desire to achieve their goals. This must be done consciously. If a habit needs practicing then all of its action sub-habits are performed consciously. But if an action habit has been learned, then once started, it is done subconsciously. It stops when it comes to the end or fails to get any of its expected feedback stimuli.
Thinking is the mental simulation of a sequence of experiences. Thinking one step ahead is the simplest form. This is the minimum thinking necessary to decide whether to perform an action habit. The process begins with the occurrence and recognition of and attention to a candidate trigger stimulus. An action habit with this trigger is found in memory. The single step thought is the recall of its goal stimulus. Based on the desirability of this thought about stimulus the action habit will be done subconsciously, not done or practiced.
Thinking more than one step ahead is directed by the desirability of goal stimuli. Each goal stimulus that is thought about is matched against an action habit trigger stimulus. This is continued until a desirable goal stimulus is reached. At this point the thought about action sequence is started.