Maja J. Mataric and social robots

In this article we expose some interesting ideas by Maja J. Mataric (University of Southern California) about the design of social robots (see Mataric, 2002 in Encyclopedia of cognitive science).

Building sociable robots includes many facets, like imitation, social learning and emotion. We can design social robots applying ideas from developmental psychology, for example and so looking for inspiration in Neuroscience. Mataric (1992) described the work with TOTO, a mobile robot being able to represent landmarks in the environment. TOTO was representative of an Artificial Intelligence interpretation of the organization of the rat hippocampus. As an alternative, Nicolescu and Mataric (2002) designed a hierarchical behavior-based architecture enabling behaviors to represent more abstract concepts. Here representations are stored in a distribuited fashion. The same perspective on generating behavior has been successful with groups of robots. This area is known as "swarm robotics". Truly coordinating a set of robots is complicated problem. Multi-robot coordination involves communication and selection action, between several tasks. In 1995 Mataric worked with the NERD HERD, a group of 20 autonomous mobile robots with limited sensing and computational abilities. Each robot was programmed with a small set of behaviors: homing, wandering, following, aggregation and dispersion. The basis behaviors were designed to conserve energy by minimizing interference between robots. Mataric (1997) described the problem of learning social rules in order to maximize energy. Robots acting within a social setting have additional sources of information: observation of a peer performing a successful action, etc. Models of people´s natural social interactions are relevant for robots in human environments. For humanoid robots this can take the form of learning natural human skills. Mataric is an excellent searcher looking for solutions which imply the design of social robots.

Affect Control Theory and Social Models of Human-Computer Interaction

One of the most interesting models of emotion is Affect Control Theory (ACT) (Heise, 1987). ACT expresses how social events are construed positing a relation between cognition and emotion. In contrast to psychological theories of social cognition, ACT emphasizes that meanings are culturally shared and deviations from meanings generate arousal that triggers re-appraisals. In this brief article and following to Troyer (2004), an outstanding specialist, I expose some aspects of the theory and how Lisa Troyer applies its concepts to model Human-Computer Interaction (HCI).

ACT looks at individuals as agents seeking consistency across interactions. Humans are categorized into roles with shared expectations regarding actions appropriate for the role. Actions in response to one another are markers of social events. Social events are formed by actors who assume identities, behaviors of actors and objects to whom the actions are directed. For instance, a policeman helping citizen, whose linguistic structure would be "Policeman Helps Citizen". Its meaning is defined in three dimensions: Evaluation (goodness), Potency (powerfulness) and Activity (liveliness). Ratings of each element are called "fundamental sentiments" (for it the word affect). Humans have culturally shared fundamental sentiments and expectations. So, we expect good policemen to behave in right ways. When elements combine in an event, emotion signals the corespondence between the meanings we expect and the actual meanings evoked. Smith-Lovin (1987) introduced impression-formation equations combining ratings of elements to estimate new ratings for inputs combined in events. These equations predict how meanings shift as interaction evolves. The sum of the squared differences between fundamental sentiments of the elements and impressions from the event generates the perceived likelihood of an event.

ACT includes a database of ratings for thousand of elements and modifiers (emotion labels for roles, for instance, "pleasant policeman"). Using the equations and database, ACT predicts the basis for expectations in subsequent interaction. The models and database are combined in the software INTERACT. With this software researchers simulate events and generate testable predictions regarding sequences and redefinitions of events. ACT has focused on Human-Human interaction but Troyer (2004) demonstrates how can be used to model Human-Computer interaction. ACT does not require that computers exhibit emotions, but only that they be able to reason about them, that is, metareasoning.

Troyer ("Affect control theory as a foundation for socially intelligent systems", 2004) designed an experiment with 15 subjects, providing them independent ratings for elements of Human-Computer interaction: "Computer", "Run Analysis", "Provide Output", "Freezes" and "Runtime Error". Correspondent social concepts were "Academic", "Ask about Something", "Educate", "Beg" and "Laughs At". Troyer substituted the correspondent social concepts for HCI terms to simulate events representing Human-Human interaction analogs of HCIs. The simulations explored how meanings shifted when a computer initially behaved as expected, producing unexpected behavior. The simulations showed how different events produced different definitions of the actor eliciting the behavior (grouch/delinquent) and the responses to that actor (scold/avoid). Besides, ACT predicted the emotions of the object receiving the behavior.

ACT may provide an architecture for designing socially intelligent systems.

Physarum machines

A Physarum machine is a programmable amorphous biological computer experimentally implemented in the state of slime mould Physarum polycephalum. It comprises an amorphous yellowish mass with networks of protoplasmic tubes, programmed by spatial configurations of attracting and repelling gradients. It feeds on bacteria, spores and other microbial creatures. When foraging for its food the plasmodium propagates towards sources of food particles, surrounds them, secretes enzymes and digests the food. The plasmodium is considered as a parallel computing similar to existing massive parallel reaction diffusion chemical processors. Adamatzky (University of Bristol) and collaborators demonstrate how to create experimental Physarum machines for general purpose computation: plasmodium can implement the Kolmogorov-Uspensky (KUM) machine, a mathematical machine in which the storage structure is an irregular graph. KUM is defined on a labeled undirected graph with bounded degrees of nodes and bounded number of labels. KUM executes several operations on its storage structure: SELECT an active node (that is, occupied by an active zone) in the storage graph; SPECIFY the node´s neighborhood; MODIFY the active zone by ADDING a new node with the pair of edges, connecting the new node with the active node; DELETE a node with a pair of incident edges; ADD/DELETE the edge between the nodes. A program for KUM establishes how to REPLACE the neighborhood of an active node with a new neighborhood, depending on the labels of edges connected to the active node and the labels of the nodes placed in proximity of the active node. In Physarun machine, a node of the storage structure is represented by a source of nutrients, an edge connecting two nodes is a protoplasmic tubes linkink two sources of nutrients corresponding to the nodes. Finally, an active node is domain of space (which may include nutrient sources) occupied by a propagating pseudopodium. The computation is implemented by several active zones. It uses distributed local sensory behaviours, approximating phenomena observed in Physarum, like foraging for food stimuli, amoebic movement, network formation, network minimisation, surface area minimisation, shuttle streaming, spatially distributed oscillations or oscillation phase shifting. The emergent plasmodium behaviours are represented taking a multi-agent approach or based upon particles. In fact, movement and internal oscillations of the plasmodium reflect the collective behaviour of the particles population. The movement of agents correspons to the flux of sol within the plasmodium. Cohesion of the plasmodium arises due to the agent-agent interactions and movement of the plasmodium is generated by coupling the emergent mass behaviours with chemoattraction to local food source stimuli. Agents both secrete and sense approximations of chemical trails being the population represented on a two-dimensional discrete map. The strength of the projected food sources can be adjusted using parameters and when the plasmodium engulfs a food source the stimulus for diffusion is reduced by the encapsulation. The diffusion gradient corresponds to the quality of the nutrient and substrate of the plasmodium´s environment, and differences in the stimulus strength, stimulus area, affect both the steepness, and propagation distance of the diffusion gradient and affect the growth patterns of the virtual plasmodium.

The Physarum machine by Adamatzky is a very interesting example of a green computer, showing the necessity for simulating simple behaviours, in first place, as motivation for developing more complex simulations. It will be an excellent source of ideas for anyone who is inspired by emerging non-silicon computers.

Pandas: Pandaemonium-Controlled Animats

(Artificial Life, vol. 16, 1, Winter 2010, The MIT Press; cover by Philip Beesley´s Epithelium, an installation at the Siegel Gallery at Pratt Institute of Design in Brooklyn, N.Y., 2008).

On this month I would like to expose an interesting model of cultural transmission developed by Chris Marriott, James Parker and Jörg Denzinger at University of Calgary. It is called PANDAS and simulates the effects of an imitation mechanism on a population of animats (artificial animals) capable of individual ontogenetic learning.

Let be a world inhabited by animats and consisting of a discrete grid that contains besides the following objects: food, water, cave and tree. The world obeys several rules of evolution: (a) no object can occupy the same cell as a tree; (b) all objects are stationary except pands; (c) water, caves, and trees are static and (d) food is depleted when used, and grows over time. For the pandas, we have these rules: (1) pandas can move one cell in one of eight directions (N,NE,E,SE,S,SW,W,NW); (2) pandas obtain food energy, water energy, rest energy and have a health parameter and, in every round, lose a fixed amount of food energy, water energy and rest energy; (3) pandas die if any parameter in (2), drops below cero; (4) a dead panda is removed from the grid and replaced with food; (5) a panda can only mate with another mating panda; (6) a panda can create a new panda by mating or spawning but a panda can only mate or spawn when it has an excess of all energy types; (7) pandas can only interact with objects in the same cell; (8) a panda can fight with another panda, reducing the target panda´s health and (9) a panda´s health automatically recovers, at an energy cost.

A panda has only a one-cell perception range and has a limited internal sense allowing it to monitor its energy levels. All perception is done using input daemons. Daemons are similar to nodes in a network playing a specific role. The input daemons that form the input layer of the panda include 9 daemons for each object in the environment (see food, see water, see cave, see tree). There is one daemon for each of the cells in the perceptual range of the panda. The input layer contains the following set of daemons (food energy, water energy, rest energy, hungry, thirsty, tired). The cognitive cycle of a panda begins with taking input from the environment and activating input daemons, and ends with an action selected. When an action has been selected, it is executed in the system and the cycle begins with the new situation.

The model is very interesting because pandas can engage in three types of adaptation: the genome of a panda encodes both "physical properties" (the panda interacts with its environment) and "mental properties" (the panda maintains its internal organization). To study the effects of the imitation on the populations of pandas, it is considered a group of pandas without imitation drive and unable to perceive directly and a second group of imitating pandas. The authors use the life span of the pandas as an indicator of their success. To do this, they introduce a parameter called the elder age, that is, an arbitrarily selected age such that when a panda reaches this age before dying, is considered successful and the value is used for the measurement of frequency. According to Marriott, Parker and Denzinger, the data of the experiments using this model, show that the median frequencies for imitating pandas are hogher than the frequencies for non-imitating pandas. Obviously, the imitating pandas have the tendency to group together, being particularly strong in newborn pandas. The model supports that mechanisms of cultural transmission can increase the frequency of success in a population but the authors look for extending it to the more sophisticated mechanisms of the "true imitation" because they think that the attribution of higher-order intentionality to artificial agents is something pending nowadays.

Gaze following and mirror neurons

Gaze following is a basic component of the human social interaction and it is a type of attention sharing behaviors. It is also present in a number of other species (for instance, apes) and seems necessary for designing social robots. It can be defined as the ability to look where somebody else is looking. Triesch, Jasso and Deák (2007) have formulated a computational model of gaze following by means of ideas related to the behavior of mirror neurons. The authors emphasize the role of learning processes by means of the interaction with the social environment. In fact, gaze following can be linked to imitation. The link between gaze following and imitation also implies the similarity between the neural basis of gaze following and the neural basis of other imitative behaviors. Triesch and collaborators develop a model that share properties with mirror neurons, that is, neurons implicated in imitation and originally founded in macaque area F5 by Rizzolatti and his team in Parma University. The model predicts the existence of a new class of mirror neurons for looking behaviors that has not been observed experimentally. In this model, an infant and a caregiver interact with a number of visually salient objects. During the process, the infant learns to predict the locations of salient objects based on the looking behavior of the caregiver. There are periods when the caregiver is present and periods when the infant is alone with the objects. When the caregiver is present, the infant and caregiver are in fixed locations facing each other with a separation between them. At any time a random number of objects will be present. Habituation decreases the perceived saliency of an object.

The infant model learns through a reinforcement learning scheme. The learning process tries to optimize the infant´s policy, that is, the way the actor maps sensory states onto different gaze shifts in order to maximize the long-term reward obtained by the infant. Reward is obtained as the saliency of the position to which attention is directed after a gaze shift has been made. At each time step, the caregiver looks at the most salient object, where saliency is mediated by the same habituation mechanism as in the infant´s visual system. The model neurons in the pre-motor layer share many characteristics with classical mirror neurons. A unit in this layer will be active during the execution of a gaze shift to a certain location in space. This is because the probability of performing such a gaze shift is related to the activation of the unit. The units in the layer will be active when the infant observes the caregiver looking in the corresponding direction. Clearly, the neurons in the pre-motor layer can be viewed as mirror neurons because the combination of being active during execution and observation of a motor act is the defining characteristic of mirror neurons.

Following to the authors, this model can be considered a simple associative learning account of a response facilitation but also has implications for the question of whether mirror neurons are innate or whether they acquire their properties through a learning process. For the mirror neurons concerned with grasping, they find plausible that there are situations where observing an agent grasp an object may predict a reward if the same action is attempted. Such situations may be sufficient for the emergence of mirror neurons for grasping. The reviewed model predicts a very close connection between mirror neurons and imitative behaviors.

Dynamic Neural Fields and cooperative robots

Dynamic Neural Fields formalize how neural populations represent the continuous dimensions characterizing movements, perceptual features and cognitive decisions of agents. Neural fields evolve dynamically generating elementary forms of cognition. Many of social activities are based on the ability of individuals to predict the consequences of other´s behavior. We have to interpret actions of our partners in collaborative works.

Erlhagen et al. (2007) try to understand motor intentions for building artificial agents using Dynamic Neural Fields. The authors consider that action understanding relies on the notion that the observer uses its motoric abilities to replicate the observed actions and its effects. They are inspired in activity patterns of mirror neurons in prefrontal cortex that postulate a chain between neurons coding motor acts. Depending on the specific chain of mirror neurons activated by contextual and external cues, the observer will predict (in a probable manner) what the observed agent is going to do. Erlhagen and collaborators represent the activity of neural populations encoding different motor acts and goals by means of Dynamic Neural Fields. The synaptic links between any two populations in the network is established using a Hebbian learning dynamics.

Let think in autonomous robots which interact in the context of a join construction task and let be a simple reaching-grasping-placing scenario. An observing robot R1 has to select a complementary action sequence depending of the inferred action goal of the other robot R2, or partner robot. So, R2 may grasp an object to place it in front of R1 with the intention to hand it over. Neural populations in the action observation layer and the action simulation layer encode motor primitives such as grasping. Such neural populations in the goal layer are associated with the respective chains in the action simulation layer. To model the dynamics of the different neural populations is used a discrete version of a dynamic neural field. Each dynamic field represents a population of 2N neurons which diverge into an excitatory and an inhibitory subpopulation, each of dimension N. The activation of an excitatory and an inhibitory neuron i at time t is governed by a coupled system of differential equations. The firing rate and the shunting term for the excitation, are non-linear functions of sigmoid shape and the interaction strength between any two neurons within the subpopulation is established by fixed synaptic weight functions which dicrease as a function of the distance between the neurons. A Hebbian learning rule for increasing the synaptic efficacy between presynaptic and postsynaptic neurons is given.

For establishing the chains, the authors employ a learning by observation paradigm in which a teacher demonstrates the sequences, each composed of motor primitives. Once the neural population becomes active, the activity propagates to all synaptically coupled populations. But only a population defining a particular chain will reach a suprathreshold activation level.

The simulation of Erlhagen et al. 2007, shows that the neural representations implementing mechanisms like motor simulation and cue integration may emerge as the result of real-time interactions of local neural populations. But, more important is that they speculate that learning by imitation takes place in two steps: first, the links between chain elements are established allowing a fluent execution of particular action sequences. In a second place, similar action sequences may have a different outcome and then, the focus shits towards the links between the goal and the contextual cues. These consequences will be crucial in the cooperative robotics domain.

REFERENCE
(http://www.mct.uminho.pt/erlhagen/pdfs/ErlhagenEtAl_ICDL07_0004.pdf)

Mirror neurons and synchronization between robots

The discovery of "mirror neurons" in the ventral premotor cortex of the macaque monkey by Rizzolatti and colaborators has generated a genuine impact in Neuroscience. In humans it is impossible to registry the neurophysiological activation of simple neurons but disregarding criticisms (see Alison Gopnik-http://www.slate.com/id/2165123/pagenum/all/-), the influence in many fields of the knowledge (study of social relations, robotics, programming, etc.) is enormous. Privileged witness is the recent work of Barakova, Lourens and Yamaguchi. Barakova and Lourens (2008) design a setup for synchronization and turn-taking behaviour in robots. For it, they connect some aspects of the neuroanatomy of the mirror system in humans with an oscillatory dynamics for neural networks.

In brief, the frontal motor areas receive sensory input from the parietal lobe. Another area is situated in the rostral part of the inferior parietal lobule. Both regions form the mirror neuron system. Besides, the posterior sector of the superior temporal sulcus form a core circuit for imitation. Modelling of the superior temporal sulcus area can be reduced to the influence of the inhibitory neurons, projecting the sensory signals to the inferior parietal lobule, area which is associated with multisensory integration. Each robot has 8 range sensors projecting to the sensory integration area that resembles the functionality of the joined temporal sulcus-inferior parietal areas. The two wheels of the robot project to the sensorimotor integration area, resembling the function of the ventral premotor cortex. Self-organization of rhythmic activity of this system can be simulated by means of endogenous oscillators an so, the change of rate of the phase with the time, is the cycle of the limit cycle oscilation, being the phase periodic over the range.

The experimental setting conceived by Barakova and Lourens presents, in the first place, robots that are taking the role of the follower, in order to establish "mirroring" couplings between the nets that simulate the inferior parietal lobule and premotor ventral areas. Hebbian connections between these nets are modelled such that the interaction behaviour is reflected by the average activation values of unikts over a certain time interval. So, the robot playing the role as a follower, tends to synchronize its motion direction with the motion direction of the leading robot. In the second place, the experiment shows the emergence of turn taking between the robots. The role of the robot, being follower or leader depends on which robot is within the visual field of its partner. The emergent turn taking is expressed by symmetry breaking process after a period of synchronization: the leading robot can become a follower and later the lead can be taken over by it.

Inspired by the mirror system in human beings, Barakova and Lourens simulate very interesting interaction behaviours of following and turn taking such that the mirroring functionality is obtained by means of the selforganization of synchronized neural firing in two robots that share perceptual space. No doubt that many extensions of this work remain to develop and promise great advances in the area of robotics.

The e-pucks and social cooperation

The e-pucks are mobile robots developed at the École Polytechnique Fédérale de Lausanne (EPFL). The designers (see Mondada et al., The e-puck, a Robot Designed for Education in Engineering, 2009) looked for the miniaturization of a complex system combining desktop size and flexibility. These robots have sensors in different modalities (distances to objects by means of eight infrared proximity sensors, accelerometer, color camera, microphones), actuators with different actions on the environment, wired and wireless communication devices and two types of processors (general purpose and DSP). Although they were conceived for education in Engineering, they have demonstrated to be very useful for experimentation in Artificial Intelligence.

Social learning is the capability of an organism to learn by observing the behavior of a conspecific. Evolutionary robotics is a methodological tool to design robots´controllers. Recently, Miglino, Ponticorvo and Donetto (2008)

(http://www.nac.unina.it/images/pub/cooperationincorvid.pdf)

have used an evolutionary algorithm for the neural control of e-puck robots which imitate the cooperation in corvids to obtain a reward (food), which is clearly visible, but not directly reachable. The dyad gets the reward if the two tips of a string are pulled at the same time.

Two e-puck robots are situated in an environment consisting of a square arena and of a corridor both surrounded by walls. The authors use an evolutionary algorithm to set the weights of the robots´neural controller. The initial population consists of 100 randomly generated genotypes that encode the connection weights of 100 corresponding neural networks. Each genotype is translated into 2 identical neural controllers which are embodied in 2 e-pucks. The 20 best genotypes of each generation are reproduced by generating 5 copies each, with 2% of their bits replaced with a new randomly selected value. The evolutionary process is iterated 1000 times and the experiment is replicated 20 times each consisting of 4 trials with 4 different starting positions in the corners of the room. Cooperation between e-pucks is regulated by social interaction with communication as a medium. The emergence of communication leads to a coordinated cooperation behavior similar to cooperation observed in corvids.

Nowadays artificial evolution is employed to build neural mechanisms that control the behavior of learning robots. Mechanisms for social learning in organisms can be successfully simulated in an integrated neural network architecture. E-pucks are an excellent tool for simulating social learning in organisms requiring cognitive mechanisms. But future work is needed, in particular to allow the robots to autonomously decide when to use social learning strategies or individual strategies.

Goran Trajkovski on imitation to modeling agents societies

In this brief review we analyse some ideas about simulation of imitation by means of artificial agents. Dr. Trajkovski is Director of Cognitive Agency and Robotics Lab in Towson University (USA). He is specialized in Cognitive Engineering and has published books like "An imitation-based approach to modeling homogenous agents and societies" or the recents "Handbook of Research on Agent-Based Societies: Social and Cultural Interactions" or "Handbook of Research on Computational Arts and Creative Informatics". Trajkovski introduces agents capable of performing four elementary actions (forward, backward, left, and right) and of noticing 10 different percepts. Each agent is equipped with one food sensor and has one hunger drive. When the hunger drive is activated for the first time, the agent performs random walk during which expectancies are stored in the associative memory. When food is sensed, expectancy emotional context is set to a positive value. Every time in the future the hunger drive is activated, the agent uses the context values of the expectancies to direct its actions. It chooses the action that will lead to expectancy with maximum context value.

Agents inhabit a two-dimensional world surrounded by walls and populated with obstacles. Sensing another agents takes the agent into its imitating mode. The agents have begun to inhabit the environment at different times. They are also being born in different places in the environment. While inhabiting the environment, they explore different portions of the environment that may be very different, or perceptually similar. In environments inhabited by heterogeneous agents, the fundamental problem is the problem of interagent communication. According to Trajkovski, an example of interagent communication is the phenomenon of multilingual agents that can serve as translators. The author proposes an enactivist (Varela, Thompson and Rosch, 1991) representation model, based on the treatment of agents as dynamical systems. The agent during the interaction with the environment generates its inputs and makes a mapping from the continuous domain of the inputs to the discrete domain of the percepts. The sequence of these percepts would reflect the structure of the environment. The basic idea is to divide up the set of possible states into a finite number of pieces and keep track of which piece the state of the system lays in at every iteration. Each piece is associated with a symbol, and in this way the evolution of the system is described by an infinite sequence of symbols. The agent generates symbols or percepts.

Trajkovski shows that imitation is far from a trivial phenomenon and that humans are wired for imitation by means of the research in multi-agent systems. He gives a solid and fertile attempt to establish a mechanism of interagent communication in the multi-agent environment using classical algebraic theories and fuzzy algebraic structures. His contributions are very relevant for the social cognition, mixing studies about animal intelligence (Thorndike) with studies about imitation in humans.

A New Computational Model of Social Learning

We describe a very interesting computational model of social-learning mechanisms proposed by Lopes, Melo, Kenward and Santos-Victor (2009). The authors distinguish between imitation ("adhere to inferred intention, replicate observed actions and effect") and emulation ("replicate observed effect") taking into account the agent´s preferences for different actions and the information available from the demonstration. There is a module addressing the baseline preferences of the agent, evaluating actions in terms of energy consumption. With this module is associated the utility function Qb ranking possible action sequences according to their overall energy consumption. A utility function Qe evaluates the actions in terms of their probability of reproducing the observed result/effect. For the intention replicating module, the utility function Qi assumes the demonstrator is goal-oriented. The module operates by considering all the possible goals in the current system, calculating for each one the relative probability that it would give rise to the demonstrated behaviour, and choosing the one that maximises the probability. In cases that are equally likely to produce the observed demonstration, goals with tied probability are ranked randomly which leads to stochasticity in the final behaviour.

The model of Lopes and collaborators can replicate the tendency to interpret and reproduce observed actions in terms of the inferred goals of the action in line with the experiments of Malinda Carpenter and colleagues (2005): a demonstrator moved a toy mouse across a table from one point to another; in one condition, the final point of the move was inside a little house, and in the other condition, no house was present: infants showed a much grater tendency to replicate the specific mouse moving action observed when there was no house to move the mouse into. The results of the simulation reproduce the findings of Carpenter et al. (2005), confirming the standard interpretation of the experiment: the infant infers what the demonstrator´s intention was.

Other simulation replicates the famous experiment of Meltzoff (1988) in which 14-month olds were exposed to a demonstrator who performed unusual actions on objects (there was a box with a panel that lit up when the demonstrator touched it with his forehead, and most infants copied the use of the forehead rather than using their hand): the infants reproduced the actions with a delay of a week. According to Lopes and collaborators, the simulation confirms the imitation in terms of the inferred intention and of sensitivity to the constraints on the demonstrator.

To investigate what happens when the learner does not have complete knowledge of the world dynamics, they model a type of experiment based on Horner and Whiten (2005) experiment in which presented preschoolers and chimpanzees with two identical boxes, one opaque and one transparent. The demonstration consisted of inserting a stick into a hole on the front of the box, with the latter step generating a reward. The insertion of the stick into the top hole was unnecessary in order to obtain the reward, but the causal physical relations were only visible with the transparent box. The results showed that 3 and 4-year-old children imitated both actions no matter whether they had observed and were tested on the transparent or opaque box, but chimpanzees were more able to switch between emulation and imitation after having observed demonstrations with a transparent box and reduced tendency to insert the stick into the ineffective hole.

The simulation designed by Lopes, Melo, Kenward and Santos-Victor replicate the results from both children and chimps when the weight of the intention replicating module was increased, confirming that the difference occurs because chimps are primarily motivated to select the most efficient method they know to achieve the end effect.

Following the taxonomy proposed by Call and Carpenter (2002), Lopes et al. (2009) build a unifying mathematical model of types of social influence on behaviour, mainly imitation and emulation, concluding that a switch between imitation and emulation might be triggered by changing the value (to the learning) of the social interaction or of the effect. So, the greater utilization of imitation by children might be explained by a stronger focus on others´intentions, mediated by social cues.

Imitation, Maslow´s Pyramid and Multi-Agent Systems

We review an interesting article published by Le Guen Herve and Moga Sorin in Proceedings of International Joint Conference on Neural Networks (2009).

The authors study the influence of imitation within a population of artificial agents following Maslow´s Pyramid of needs. Imitation is the possible origin of communication and social learning. The imitative abilities include many types like facial imitation, perception by the infant that he or she is being imitated and empathy. The interest in empathy induces the study of emotions which Herve and Sorin link to the analysis of social needs as modeled by the Maslow´s Pyramid of motivations. Although the influence of each particular need varies from one person to another the principle is there are two main classes of needs. The most important needs are called primary needs (eating, breathing, etc.) while the second class of needs constitute the social aspect of human motivation and includes the need for esteem. This need is bidirectional and has been interpreted by the authors as the need to imitate and the need to be imitated. As the fact of being imitated is perceived by the agent concerned and as imitation is likely to generate empathic satisfaction, the authors model them as expected rewards. Based on a model of an autonomous robot with goal-oriented navigation and imitation capabilities, the robot goals are derived from internal variables that have to be maintained in a comfort area. The values of these variables decrease in time. The robot population´s task is to explore an unknown environment and to localize sources corresponding to its needs. Its survival will depend upon the satisfaction of these needs by discovering the different locations of the sources. The behaviors are obstacle avoidance, goal-oriented navigation, imitation and exploration. The emotional signature expresses the current state and the expected state of the agent: an agent in its comfort area displays a neutral signature whereas an internal variable below a certain threshold induces pain. That pain will cause a potential empathic response that is likely to incite another agent to move toward a known source. Besides a motivated agent is likely to provoke an attractive empathic response: keeping a motivated agent in its own field of perception becomes a source of motivation. It leads to the selection of the imitation target according to its motivational state.

The results of the simulation with Maslow agents showed an enhancement of the global survival rate even with a very small population wherein communications are not frequent.

The authors have proposed a holistic approach to the implementation of imitation in autonomous agents.

We recommend this excellent article because its ideas permit to build a simple and scalable model of agent useful in applications such as networks or swarm piloting.

Agent Societies and Artificial Intelligence

Very recent articles have addressed the problem of the interaction between populations of virtual robots. Vijayakumar and Davis ("Metacognition in a "society of mind"") investigate the concept of mind as a control system using the "Society of Mind" idea from Marvin Minsky. They develop Metacognition in a model based on the differentiation between metacognitive strategies. Metacontrol is a part of metacognition. The authors explore Metacognition mechanisms in developing optimal agents for the fungus world testbed. The fungus world environment allows the behaviours of a robot to be monitored, measured and compared. It is generated a swarm intelligence of how the group of agents work together to achieve a common goal. Vijayakumar and Davis implement an architecture with six layers including reflexive, reactive, deliberative, learning, metacontrol and metacognition layers. The authors design an experiment with different types of agents (random, reflexive, reactive...) All agents move in the environment, changing direction in case of obstacles. To compare the results of each agent, the following data were collected: life expectancy, fungus consumption, resource collection and metabolism. Total performance was denoted by the combination of resource collected and life expectancy. Experiments were conducted for each type of agent. In Simulation 1, deliberative agents collected 50% of resource and left 64% of energy. In Simulation 2, the results of deliberative agents were compared to Metacognition agents, collecting metacognitive agents 82% of resource and lefting 71% of energy. The result concludes that Metacognition agents are better than other cognition and deliberative agents. Thus, Metacognition is a very powerful tool for control and self-reflection and intelligent and optimal agents can be viewed as collective behaviours as a "Society of Mind". Really to develop a Metacognition concept in Artificial Intelligence seems necessary for building self configurable computational models and true intelligence.

MathPsych 2009 (Amsterdam)

The Annual Convention of the Society for Mathematical Psychology ("MathPsych 2009") was held in Amsterdam on August 1-4, 2009. We stress the highlights for the purpose of our blog, Social Cognition.

On August 2, Joe Johnson (Miami University) illustrated, via a simple mathematical model, the ability for predicting decision behavior based solely on perceptual data. Johnson uses an evidence accumulation model with a ratio choice rule to predict athletes´intuitive, initial choice in a realistic game situation. Markus Raab (German Sport University in Cologne) investigates the preference for intuitive decisions in contrast to deliberative ones. He applies a mathematical choice model based in team handball attack situations. Bayesian models also are applied and so Tom Lodewyckx (University of Leuven) and others design a Bayesian state-space model for affectivity. They use Markov chain Monte Carlo methods to estimate the model parameters. This framework is used to high resolution psychophysiological and behavioral data obtained during adolescent-parent interactions expressing dynamical emotions.

Vanpaemel (Umiversity of Leuven) and Michael Lee (University of California, Irvine) advocate the advantages of hierarchical Bayesian modeling in providing one way of specifying theoretically-based priors for competing models of category learning. In a similar line, the group of Rich Shiffrin (Indiana University) spoke about information integration in perceptual decision making. According to the authors, researchers studying judgment and decision making have shown that people employ sub-optimal strategies when integrating information fron multiple sources. But another group of researchers has had success using Bayesian optimal models to explain information integration in fields such as perception, memory and categorization. Shiffrin and colaborators design a decision making experiment to test the range of this difference.

Amy Perfors and Daniel Navarro (University of Adelaide) consider the situation in which a reasoner must induce the rule that explains an observed sequence of data, but the hypothesis space of possible rules is not explicitly enumerated or identified. They present mathematical optimality results showing that as long the hypotheses tend to be sparse (that is, tend to be true only for a small proportion of entities in the world), then confirmation bias is a near-optimal strategy. The authors propose, in a very interesting manner, to chose queries that one knows will lead to an affirmative response for at least some hypotheses (hypotheses being considered). This positive-test strategy is closely related to the confirmation bias.

The Meeting finished on August 4 hoping to achieve the same successful objectives for coming events.

Public Good Games and Social Cognition

Public good games provide an interesting testbed for the study of social dilemmas. Social dilemmas are inherent in decision making. Despite the diversity of dilemmas, all share the same structure: each individual benefits from behaving selfishly but a group gets greater rewards if its members cooperate.

In a typical public good game, players may divide their initial endowments into both a private account and a group account. The frequent strategy for a rational player is defection but most participants in public good experiments do cooperate to some extent. Factors that enhance cooperation are the communication among players or the payoff structure of the game. Nowadays it is thought that behavioral investments could be an important determinant of cooperation in social dilemmas. Participants´ endowments are distributed asymmetrically among a group to discover whether "rich" individuals will contribute more than poor individuals. The conclusion is that the differences in the level of contributions may be reduced if positions are assigned on the basis of merit rather than chance. Although asymmetries in participants´ behavioral investments (for instance, effort investments) affect fairness judgments and group identification, members are often not aware of such differences. But the origin of the endowments affects contribution levels in the public good game (Muehlbacher and Kirchler, 2009). Subjects who earned their endowments through a greater amount of effort were less cooperative than those individuals who had earned the money with ease.

Chaos Theory and Social Cognition

We will review the last contributions of the theory of nonlinear dynamical systems to the field of social cognition, specially

Vallacher and Nowak (2009).

Social relations evolve and change in the absence of external influences. Psychological systems display intrinsic dynamics. Three basic types of attractors have been identified: fixed-point attractors, periodic attractors and deterministic chaos. A fixed-point attractor describes the case in which the the state of the system converges to a stable value. It is similar to the notion of homeostasis. It corresponds, for example, to a desired goal. Multiple fixed-point attractors express that people can have different (perhaps contradictory) goals and patterns of social behavior.

Some systems display oscillatory behavior. A temporal pattern showing this tendency is a periodic attractor. Social judgement often oscillates between positive and negative assessments.

Chaos represents a possibility in many social phenomena. Social psychology presents many nonlinear phenomena, such as complex interactions among variables or inverted-U relations.

Social interdependence is very important to game theoretic approaches in Psychology, for instance, the prisoner´s dilemma game. So, Nowak et al. (1990) used cellular automata to model the dynamics of social influence. Each individual had three properties: an opinion on a topic, a degree of persuasive strength, ans a position in a social space. In each round of the computer simulations, one individual was chosen and influence was computed for each opinion in the group. The updating rule was the following: the individual changed the opinion to match the prevailing opinion if the resultant strength for this opinion position was greater than the strength of the individual´s current position. The minority opinion survived by forming clusters of like-minded people.

The dynamical account of social influence describes how the state of a single individual depends on the state of other individuals. However, individuals are best conceptualized as displaying patterns of change rather as a set of states. Social influence can be approached as the coordination over time of individual dynamics. Individuals in a relationship are represented as separate systems capable of displaying rich dynamics. A recent model of synchronization has been developed by Nowak, Vallacher and Zochowski (2002). Coupled logistic maps are used in this model. The behavior of each individual not only depends on the preceding state but also on the preceding state of the other person.

Dynamical social psychology has generated a deep source of formalisms but social reality can not be confused with physical reality. Individuals are not interchangeable in the way that atoms are. People live in a symbolic world and do not respond in a reactive way to the objective features if the environment. For it, human dynamics contain some degree of randomness, and human behavior is often unpredictable and perhaps chaotic.

Clint Sprott (Fractal image)

Peter Gollwitzer and social cognition

I would like to pay homage to Peter Gollwitzer, presenting some of his outstanding contributions to social cognition.

Peter Gollwitzer represents the great German tradition in Psychology: from Wilhelm Wundt to Kurt Lewin, from Asch to Heinz Heckhausen, Germany always has brought excellent researchers to the study of the volitional processes. Gollwitzer is renamed expecially by his contributions to intentional behavior studying implementation intentions. Intention implementations are currently at the center of research in the world of the Psychology. By means of implementation intentions, bringing a goal pursuit to a successful end is facilitated. They are effectives in different areas such as health domain or academic performance. So, the likelihood of performing a breast revision was enhanced by forming implementation intentions. Besides, the effects of implementation intentions on reducing dietary fat intake were relevant. Bad habits are controled by implementation intentions and Schweiger Gallo has verified the effectiveness of forming implementation intentions on the suppression of undesired emotional responses (for instance, fear of spiders). But perhaps more important is the study of implementation intentions effects on critical populations, such as addicts in withdrawal, schizophrenic patients or subjects with frontal brain lesions. Another clinical population who profits from implementation intentions is ADHD children: in a study, the ADHD-children who formed an implementation intention showed the same response inhibition performance as children without any psychological disorders.

But perhaps the two pending tasks is to study the design of Artificial Intelligent systems incorporating implementation intentions and the underlying mechanisms of implementation due to neurophysiological research. The first one has became to be analysed by the author of this blog in his article about implementation intentions and artificial agents. The second one is being studied by means of the electrocortical correlates of emotion regulation by ignore-implementation intentions. It seems that these intentions produce their effects through cortical control that sets in the information processing system blocking the emergence of negative emotions. Electrocortical correlates offer the possibility of determining at what point different kinds of implementation intentions (suppression or inhibition) exert their effects after the critical stimuli are encountered.

Forming implementation intentions reveals to be an effective self-regulatory instrument in domains including social psychology and clinical psychology. Gollwitzer continues the tradition.

Joint Action and Artificial Intelligence

In this article, we examine how the research about joint action (Sebanz, Knoblich, Bekkering...), can be intertwisted to the ideas about distributed coordination in uncertain multiagent systems (Maheswaran, Szekely, Rogers, Sánchez...)."Joint action can be regarded as any form of social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment" (Sebanz, Bekkering, Knoblich, 2006, p. 70). According to the authors, successful joint action depends on the abilities to share representations, to predict actions and to integrate predicted effects of own and other´s actions. A mechanism for sharing representations of objects and events is to direct one´s attention to where an interaction partner is attending. However, a more direct mechanism is provided by action observation; studies about mirror neurons show that during observation of an action, a corresponding representation in the observer´s action system is activated. On the other hand, an efficient means to predict other´s actions that is not based on action observation is knowing what another´s task is. A series of recent studies has shown that individuals form shared representations of tasks quasiautomatically, even when it is more effective to ignore one another. But how individuals adjust their actions to those of another person in time and space can not be explained just by the assumption that representations are shared; action coordination is achieved by integrating the "what" and "when" of others´actions in one´s own action planning. This affects the perception of object affordances, and permits joint anticipatory action control. Very recently, Maheswaran, Rogers and Sanchez (2007) introduce the idea of distributed coordination in multiagent systems. I believe that the research by Sebanz and others in humans can be applied successfully to the area of distributed Artificial Intelligence. Centralized systems can generate fully coordinated policies but put a very high computational load on a single agent. Given a team of agents, every agent in the team has a set of activities that it can perform. Each activity has probabilistic outcomes. Only the agent has current knowledge of its policy at all times. The team reward is a function of the qualities of all activities, and the agents´ objective is to maximize this reward at some terminal time. One way this function can be composed is with a tree where the activities are leaf nodes. Each non-leaf node is associated with an "ancestral" operator which takes the qualities of its children as input. The output of the root node is the team reward function. An agent´s subjective view of the reward function can be defined. It is considered the case where each agent see all ancestral nodes of activities they own, and any nodes and links that connect to its activities and their ancestral nodes via directional operators. The authors introduce distributed coordination between artificial agents, that is, something like the joint action in humans studied by Sebanz and others in humans. In an immediate future, a fertile cross will happen between these two roads.

Winter School 2009: Social Structures in Communication Networks

On 5-10 January 2009 and chaired by Jorge Louça, has been celebrated at the University of Lisboa, the Winter School 2009: "Social Structures in Communication Networks". On Monday, Professor Louça presented the program and scientific goals in the context of the International Doctoral Program in Complexity Sciences. The "Universidade" is accomplishing a great effort following the way pioneered by Santa Fe Institute or the Complex Science Society at Paris.

Professor Araújo analysed whether biological-inspired network models contribute to the understanding of aggregate economic behaviors. Besides, and following the ideas by John von Neumann, she explained the redundant nature rather than efficient nature of the network structures, contributing to clarify the idea of efficiency versus reliability in social environments. In the evening, Professors Louça and Rodrigues introduced tools for the study of social networks like NWB, PAJEK, GUESS or UCINET.

On Tuesday, Professor Symons introduced in his second presentation, ideas including references to the analysis of emergent properties. Symons offers an alternative to the macro-properties model of emergence. Emergence is not restricted to macro-phenomena, but can appear at the intersection of networks. Objects or agents can be involved in distinguishable systems or networks simultaneously. An agent may participate in social, economic, political and other networks at the same time.

On Wednesday, Professors Louça and Rodrigues showed multi-agent based social simulation by means of tools like NETLOGO, MASON or BREVE, while on Thursday, both Professors outlined community detection in social networks, employing algorithms like Girvan-Newmann. In the evening, Professor Palla exposed the statistical properties of community evolution in complex networks. On Friday, Professor Marinheiro spoke about network architectures and organization. Network maps have been used to mitigate the problem of the understanding of the network structures. Professor Lopes explained that in the networked multimedia realm, "more is different".

Finally, on Saturday, 10th, it was programmed a keynote talk by James Sterbenz (University of Kansas), concerning adaptive computer network architectures.

I am very grateful to the University of Lisboa by the organization of this School in which young students and prestigious Professors, have shared very interesting viewpoints about the formal and social analysis of networks, mixing complex systems and dynamics of networks.

"The brain is a network of neurons; organizations are people networks; the global economy is a network of national economies, which are networks of markets...How do such networks matter?"

The link to the Winter School is:

http://idpcc.dcti.iscte.pt/ws2009/ws2009_home.html

Synergetics and social cognition

The term "synergetics" was introduced by Hermann Haken, pioneer in the study of the laser, on 1970. It means the science of cooperation and can be considered as a science of orderly, self-organized, collective behavior subject to general laws. Therefore, the aim of synergetics is to establish the natural laws on which the self-organization of systems is based.

In physics there are different aggregate states-solid, liquid, gaseous-called phases, and the transitions between them are called phase transitions. The three phases differ only in the arrangement of the molecules. If we heat a layer of liquid in a dish from below and if the temperature difference berween top and bottom is only slight, there will be no motion of the liquid on a macrolevel. But when the temperature difference is further increased the liquid begins to move macroscopically in a quite orderly manner in the form of rolls. The curious fact is that such hot drops do not rise irregularly but in an orderly manner. Nature discovers that it can transport the heated parts upward more efficiently when they join in a regular motion. If we add the individual motions of the rolls, we obtain a hexagonal pattern. The liquid rises in the center of the hexagons and sinks along the outside:

Once the choice is made the alternatives are out of the question, and the choice cannot be reversed. Minor fluctuations decide the nature of the choice. Once it has been made, all particles must accept it. Increasingly complex motion patterns can be created by self-organization, that is, and employing the language of synergetics, new order parameters succeed each other.

Nowadays, many concepts of the synergetics, like "attractor", "bifurcation", "fluctuation", "synchronization effect", "symmetry broken"... are useful for the application to social sciences. For instance, to social conflicts. According to Haken, conflicts exist that offer two equivalent solutions in which society resolves the conflict for individuals merely displacing it. Do the courts favor the mother or the father in the child´s upbringing? (Haken, 1984). The symmetry must be broken by the judge. The advantages and disadvantages of one solution are balanced against those of the other.

Neuro-psychoanalysis: a new revolution?

In this brief article we will explain the current connections between psychoanalytic concepts, like Ego or Id, and Neurology and the computational study of the episodic memory for artificial agents. We will refer to the book by Solms and Turnbull, "The Brain and the Inner World" and to the recent work by Andreas Gruber at the Vienna University.

In 1923, Freud recognized that the rational part of the mind is not necessarily conscious: consciousness was not a fundamental organizing principle of the architecture of the mind. Freud attributed the functional properties previously assigned to the system Conscious-Preconscious to the "Ego", being conscious only a small portion of the Ego´s activities. Its main property was the capacity for inhibition. Freud thought of the capacity to inhibit drive energies, the basis of all the Ego´s rational. Consciousness over time allows the development of what Damasio (1999) calls an "autobiographical self". In psychoanalytic terms, the core "self" might be described as a perception of the current state of the "Id", whereas the autobiographical self is synonymous with the "Ego". Episodic memory is the fundamental element of the "autobiographical self". Following to Schacter (1996), the episodic memory system allows explicitly to recall the personal incidents that define the lives of human beings. Very recently, Andreas Gruber (2007) at the Vienna University, has built a computational model of episodic memory that is based on Freud´s Ego-Superego-Id personality model. A pre-decision module, representing the Freudian "Id", consists of the drives and the basic emotions modules. In this model, low-level decision making is done. If a drive is very high, the system will try to bring the drive back into its balanced range. By the other side, perceived situations are handed over to the the high-level decision making module: it represents the "Ego". Decision module interacts with the episodic memory by searching for similar situations to the current one including their emotional rating. A behavior may be triggered directly by a complex emotion while reactive responses arise from the pre-decision module, whereas routines are longer sequences of actions stored in the procedural memory.

It is very curious to verify that the Freudian paradigm, always impugned by the absence of scientific methodological rigour, currently is guiding the search in Neurology and Artificial Intelligence. Are we witnessing a new and revolutionary synthesis?

Dunlosky & Metcalfe: "Metacognition"

Two prestigious authors, John Dunlosky and Janet Metcalfe, have published on September the first introduction to Metacognition for undergraduates. Dunlosky is a Professor of Psychology at Kent State University and Metcalfe is a Professor of Psychology and of Neurobiology and Behavior at Columbia University. This book is an excellent tool for understanding in a concise way the state of the art of the emergent field of the Metacognition. From a didactic point of view, we remark the presence in each chapter of boxes presenting specific subjects and closing the chapters, questions discussion and review concepts which allow to consolidate the learning.

An introduction starts presenting the metacognitive model of Nelson and Narens (1990), the most followed model in this area. In this model, the interplay between the meta-levels and the object-level defines the two process-based activities of metacognition: monitoring and control: control is exerted whenever the meta-level modifies the object-level; controlling the object-level provides no information about the states of the object-level. For it, you must monitor the object-level activities so that you can update the model of them.

Chapter 2 presents a very interesting History about the theoretical development of the concept. Perhaps it´s the first historical compendium published. Since the modern pioneer John Flavell to Asher Koriat many names and photos of researchers are included.

Section 1 analyzes the main empirical methods for investigating the Metacognition and chapter 4 speaks about feeling-of-Knowing (FOK) judgments and Tip-of-the-Tongue states. In the first case, Dunlosky and Metcalfe introduce theories like "Target Stregth" (Hart, 1967), "Cue Familiarity" (Reder, 1987) or "Target Accessibility" (Koriat, 1993). An interesting section about the contributions by Lynne Reder and colleagues on strategy selection by means of (FOK) judgments, finishes the chapter.

Judgments of learning (JOLs) are decisive for Education. In a seminal work, Arbuckle and Cuddy (1969) argued that if paired associates "differed in associative strengths immediately following presentation, subjects should be able to detect these differences just as they can detect differences in strength of any other input signal". Their students studied short lists of paired associates, and inmediately after studying each one, they judged whether or not they would recall it. Following Dunlosky and Metcalfe (p. 94), research on JOLs has focused on contexts in which extra study typically does improve memory. Besides self-feedback from intervening test trials does benefit the relative accuracy of JOLs. There are several hypothesis explaning the effects of JOLs. Begg et al. (1989) proposed that JOLs are based on the ease of processing an item inmediately prior to making the judgment. Bjork and Schwartz (1998) have demonstrated the effect of retrieval fluency on JOLs. Koriat (1997) has used the cue-utilization approach. Acording to this perspective, people use a variety of cues-item relatedness, number of study trials...-to infer whether an item will be remembered.

Confidence judgments or retrospective confidence (RC) judgments are analysed in chapter six. What causes the overconfidence effect? Perhaps cognitive biases distort people´s judgments but the overconfidence effect is probably an artifact of experimental methods.

Source judgments (chapter seven) involve remembering the source of a memory or the context in which a memory originally occurred. Between the factors that influence source-monitoring accuracy are the similarity of the sources and emotion and imagery.

Chapter eight is a very interesting one. The authors review some findings on people´s assessment of the truth of witnesses´memories based on their expressed confidence, and on people´s abilities to detect lies. Many cases are presented specially about hindsight bias: this effect has important implications for the criminal justice system, for isues as whether jurors are able to disregard testimony that has been ruled inadmisible.

In chapter nine, are considered in detail the wide connections of Metacognition to Education and this textbook finishes including a section about life-span development. We remark in chapter ten the interesting references on development of theory of mind and metacognition. In the other side (chapter eleven), it´s reviewed the problem of aging and memory monitoring.

This book is not only the first textbook to focus on Metacognition but perhaps the first book in covering metacognitive research in an unified framework. It´s also an excellent handbook for more advanced students and has been writen by two authentic champions of the subject.

Martin Nowak and Cooperation

Martin A. Nowak, the master of the spatial games published two years ago a book, entitled "Evolutionary Dynamics: Exploring the Equations of Life" (Belknap Press, 2006). We remark some references by Nowak about cooperation in this book.

According to Nowak, experimental game theory shows that humans do not behave rationally. They are guided by instincts that might have evolved via different situations. In the Prisoner´s Dilemma, humans often try to cooperate. Cooperation means not to cooperate with the state attorney but to cooperate with your partner and remain silent. If both of you defect, both will get a long prison sentence. No matter what your partner does, it is better for you to defect. The rational analysis suggests that both prisoners will confess and spend a long time in jail. Defector always have a higher fitness than cooperators. Natural selection increases the frequency of defectors until cooperators have become extinct. But cooperation becomes a promising option when the game is repeated several times between the same two players. Consider a strategy in which an agent cooperates on the first move and then cooperates as long as the opponent does not defect. If the opponent defects once it will switch permanently to defection and it will never forgive. This strategy is a strict Nash equilibrium versus a strategy consisting in always defect. In terms of evolutionary dynamics, if the whole population uses the first strategy, then the second one cannot invade. Obviously, human strategic instincts are not formed by playing games with a well-defined number of rounds. In the lifetime there might always be another round. There is always a tomorrow in our plans.

Towards a computational model of craving in addictions

Several computational models try to explain the brain mechanisms in the origin of addictive behaviors. Perhaps, David A. Redish, Professor at the Department of Neuroscience (Minnesota University) has been the pioneer. He published in Science (2004) the article "Addiction as a computational process gone awry", a milestone for many other computational models.

Following to Redish and Johnson (2007), there are two systems in the mammalian brain with differing levels of search: (1) a flexible system, which is capable of being learned quickly, but is computationally expensive to use, and (2) an inflexible system, which can act quickly, but must be learned slowly. The flexible system allows the planning of multiple paths to achieve a goal; the inflexible system only retrieves the remembered action for a given situation. The authors hypothesize a unified system incorporating three subsystems, a situation recognition system and two contrasting decision systems-a flexible, planning-capable system that accommodates multiple paths to goals and takes into account the value of potential outcomes, and an inflexible, habit-like system.

The flexible decision-making system requires recognition of a situation, recognition of a means of achieving outcome from situation as well as the evaluation of the value of achieving outcome.

The inflexible decision-making system entails a simple association between situation and action. Evaluation entails a memory recall of the learned associated value of taking an action in the situation. To evaluate the value of an outcome, the system needs a signal that recognizes hedonic value. Two brain structures that have been suggested to be involved in the evaluation of an outcome are the orbitofrontal cortex and the ventral striatum. Neurons in the ventral striatum show reward correlates, and anticipate predicted reward. The hippocampus projects to ventral striatum and ventral striatal firing patterns reflect hippocampal activity.

It seems that hedonic signals are carried by opioid signaling. If endogenous opioids signal the actual hedonic evaluation of an achieved outcome, then when faced with potential outcome signals arriving from the hippocampus, one might expect similar processes to evaluate the value of expected outcomes. According to Redish and Johnson (art. cit., p. 330), this predicts that the effect of hippocampal planning signals on accumbens structures will be to trigger evaluative processes similar to those that occur in response to actual achieved outcomes. This has an effect for craving. Craving can be defined as the intense desire for something. Because the flexible system only entails the recognition that an action can lead to a potential path to a goal and does not entail a commitment to action, craving is not necessarily going to produce action selection. When the hippocampal component reaches a goal that is evaluated to have a high value, this will produce a strong desire to achieve that goal: the psychological effect of that recognition is to produce craving.

Competitive opioid antagonists have been used clinically to reduce craving. The hypothesis that reward signals are released on recognition of a pathway to a high-value outcome implies that blocking those reward signals would not only reduce the subjective hedonic value of receiving reward, but would also reduce craving for those rewards. If that reward signal is based on opioid signaling, then this may explain why an opioid antagonist such as naltrexone can reduce craving.

José Mira: in memoriam

On August 13th, 2008, Spanish engineer José Mira died in Madrid. Specialists in Artificial Intelligence and others will remember Mira as a sharp thinker and a nice person, whose interests and expertise ranged from theoretical and applied Artificial Intelligence to Cognitive Science. Professor Mira did much to establish in 1989 the Department of Artificial Intelligence at UNED. Mira' s research contributions were focused on methodological aspects in Knowledge Engineering and the connections between Neuroscience and Computation. From an applied perspective he worked about Knowledge Systems in Medicine and Industry. Definitely, Mira has been one of the principal influences on AI in Spain and Latin America: his death feels like losing a decisive influence.

"Fungus Eaters" and Artificial Intelligence: a tribute to Masanao Toda

In 1962 Masanao Toda published a revolutionary study that was the origin of the idea of virtual complete autonomous agents and that preluded branches very important in Artificial Intelligence like autonomous robots (Brooks, 1991), swarm intelligence (Bonabeau, Dorigo and Theraulaz, 1999) or evolutionary robotics (Nolfi and Floreano, 2001). Against a cognitive science centered in a symbolic perspective, Toda analyses systems capable of behaving autonomously in an environment without external aids. This is a very hard problem in Artificial Intelligence. Complete autonomous systems have to incorporate capabilities for deciding what to do interacting with the environment.

In his article "The design of a fungus eater" (Behavioral Science 7, 164-83), Toda "consider(s) a design of a robot to be sent to a hypothetical planet as a robot uranium miner, which sustains itself by eating fungi as its energy source". This solitary creature feeds on a type of fungus that grows on the planet and the more uranium ore collects, the more reward it will get (see also Pfeifer, 1996, 3-12). The "Solitary Fungus Eater" (SFE) has means of collection and locomotion and means for decision making interacting with the perception of the environment. It must be autonomous and self-sufficient because it can not receive external aids: it is a virtual adaptive system, a genial idea in 1961-62. Besides Toda introduces another insight: the notion of bodily designed agent. He employs a choice program for regulating the behavior of the "Eater" using stochastic distributions. Toda defines the expected amount of uranium exploitation by means of a function f indicating the expected amount of uranium exploitation as a function of fungus storage (Toda, 1982, p. 114). The problem is that f cannot be determined until the choice program is fixed and the optimal choice program is given only after the function f is specified. The key is in defining adaptability in an unknown world.

But Toda advances almost 30 years and proposes for investigating matters like emotion, irrationality, social psychology, etc., for autonomous agents. Does SFE have emotion? Of course, the answer depends upon the definition and perhaps it can be defined as a particular state of mind accompanied by a high level of energy mobilization. After Rosalin Picard will introduce the paradigm of "affective computing" and nowadays many researchers design virtual agents or robots incorporating emotions or something like that: Lola Cañamero, Eva Hudlicka, Darryl Davis, Rolf Pfeifer, Thomas Wehrle and many others (see the excellent introduction by Ruebenstrunk, 1998, http://www.ruebenstrunk.de/)

Besides Toda thinks that apparently irrational behavior of human being can also be incorporated in the SFE. Herbert Simon had spoke about bounded rationality, notion exploited after by Stuart Russell or Gerd Gigerenzer and, in the meantime, the omnipresent contribution by Toda.

And what to say about the communication and cooperation between agents? Toda (1982, p. 127) believes that "optimal cooperation programs for Fungus Eaters will provide a heuristic for the sciences of human interaction" and...for robotics? Chains of e-pucks robots colaborating between them (see Francesco Mondada in http://www.e-puck.org/) demonstrate this prophecy in 2008.

Masanao Toda passed away on 5th September 2006. Who was Masanao Toda? Was he the prophet of a new world inhabited by artificial autonomous agents?

Social Cognition in Berlin: "XXIX International Congress of Psychology"

The "XXIX International Congress of Psychology", organized by "The German Federation of Psychologists´Associations", under the auspices of the "International Union of Psychological Science (IUPsyS)" was held in Berlin on July 20-25, 2008.

On Monday, 21 July, Don Spangler in "Computer measurement of motives", described the development and validation of a computer program to measure implicit motives. This program analyses namely speeches, written materials, interviews, transcripts of meetings, and articles appearing in the business press of USA.

Hugo Kehr gave an overview of his compensatory model of motivation and volition. Structural components of the model are implicit motives, explicit motives, and perceived abilities. Its functional processes are volitional regulation (compensating for inadequate motivation) and problem solving (compensating for inadequate perceived abilities).

In "Routines and decision making", Tilmann Betsch emphasizes that empirical evidence indicates that routines influence behavior generation, information search, appraisal, choice and the implementation of a chosen behavior in human decision makers.

Professor Gao presented a computer program which can divide the critical thinking procedure into several distinctive stages, and diagnose individual logic fallacy instantly.

In the "Social Cognition I" session, Julia Herfordt spoke about social facilitation. Professor Herfordt tested Zajonc´s social facilitation hypothesis with an antisaccade task.

Professors Shi and Wang argued that two different imputations in opposite directions may occur at the starting point of the empathy process and that whether the information of imputation comes from external or internal source will determine which one really happens: the results showed that the difference of information source affected the way how participants used the information to infer what other people would know and act.

To end this paper session, Professor López Alonso analyzed social representations from their inferential bases and structures as cognitive processes.

The paper session of "Social Psychology" was opened by an interesting contribution about the possible existence of structures isomorphic to feedback cycles in social psychological phenomena; Professor Caraiani analyzed classes of symmetries and synchronization of chaotic phenomena through the consideration of the correlation between cyclic invariants.

On Tuesday, 22 July, Professor Bertrams chaired a Symposium about "Self-regulatory strength and ego depletion". According to Professor Schmeichel, both cognitive load and ego depletion undermine self-control but they don´t operate via the same mechanism.

Professor Bertrams believes that success in self-control and complex thinking depends on a resource comparable to the strength of a muscle. This regulatory strength can be boosted by regular self-control effort. In the afternoon, Ulrich Wiesmann presented his structural modelling approach for the generalized health-related self-concept. Professor Wiesmann starts out from Markus´dynamic self-concept theory and finds support for a hierarchical factor structure.

On Wednesday, 23 July, a paper session about "Self-regulation" was chaired by Professors Sassenberg and Mamali. Jan Crusius analyzed the role of spontaneous social comparisons in automatic goal pursuit. Professor Crusius suggests that social comparison is a mechanism contributing to goal priming.

Waclaw Bak specified three dimensions of self-regulation. Based on conceptions of of Higgins, Markus, and Ogilvie, the self-system is defined as a cognitive structure, composed of different self beliefs (ideal-self, ought-self, undesired-selves, can-self, impossible-self) and discrepancies between them. Professor Bak hypothesizes that the self-system one can describe in terms of three dimensions: (a) negative-self standards; (b) positive self-standards and (c) can-self-standards.

The paper session "Social Cognition II", chaired by Carlos Pelta, was opened by Professor Kwong with an oral presentation on perceptions of progress towards goal. Following to Professor Kwong, perceived progress plays a critical role in people´s motivation to persist toward a goal. She has been demonstrated that while figure displays exert dominant influences over corresponding numerical displays, this effect is dependent on the relative fluency in processing the two different modes of information.

Professor Pelta presented "Implementation intentions and artificial agents" in collaboration with Professor González Marqués at the Complutense University of Madrid. The contents of this presentation have been exposed thoroughly in this blog.

Finally, in the paper session about the impact of goals and volitional processes on learning, Katrin Jorke presented Trait Procrastination as a failure in self-regulation that can have a negative impact on the efficiency of self-regulated learning.

During the Congress, an exhibition of posters allowed to obtain relevant information about the progress of Psychology in the world. Special thanks to the Organization Committee for their professional contribution in organizing this manifestation. I hope that the XXX Congress in Cape Town will be a success.

12th Congress of the European Federation of Neurological Societies

The 12th Congress of the European Federation of Neurological Societies was held in Madrid on August 23-26, 2008. Approximately 1,800 abstracts have been submitted and more than 4,000 persons have participated. Seventeen satellite symposia were available covering the latest therapeutical and diagnostic acquisitions.

The Congress was opened by the President of the EFNS, Jacques L. De Reuck and the first course was entitled "How do I examine...?". Professors Gil-Nagel, Quinn and Clanet spoke about the diagnostic in epilepsy patients, in patients with suspected Parkinsonism and patients with a multiple sclerosis, respectively.

On Sunday 24, we emphasize a session about Neuro-estimulation for epilepsy. According to Professor Wadman, deep brain stimulation is a successful therapy for epilepsy patients. However, as it is not known what the underlying fundamental mechanism is, it is unclear where the best location for stimulation is. It is needed distinguish between acute intervention in ongoing, just started, or about to start epileptic seizure and an approach where the main goal is to chronically decrease excitability and so reduce the likelihood of the occurrence of a seizure. Another interesting aspect is the applicability of closed-loop strategies where the level of stimulation is controlled by the state of the brain. Professor Wadman thinks that in epilepsy there are reasons to assume that continuous stimulation is not necessarily the best approach.

On Sunday in the morning, Professor Barbro B. Johansson explained the fascinating matter of the brain plasticity and stroke rehabilitation. According to Professor Johansson, training in specific activities of daily living, and starting within the first week or weeks after stroke are important key factors in stroke rehabilitation. The healthy brain has a large capacity for automatic simultaneous processing and integration of sensory information. Cortical lesions interrupt networks that combine different regions, and the capacity for automatic processing of incoming stimuli is reduced. Cortical sensory and motor representation of the hand exerts inhibitory influences on the motor opposite cortex in healthy individuals. Based on the observation of an abnormally high inhibition from the intact side in patients with cerebral infarcts, has been hypothesized that this abnormality might adversely influence motor recovery.

Applicability of neuroimagen techniques has been a topic very important in this Congress. On Monday 25, Professor Berg presented imaging techniques of basal ganglia disorders. However, SPECT or PET techniques are limited because they do not discern between idiopathic Parkinson´s disease and atypical Parkinsonian syndrom. Special MRI techniques, like diffusion tensor imaging (DTI) and volumetry of the respective areas need to be applied.

On Tuesday 26, there was a session about Cognitive Neuropsychology. It was very interesting a presentation by Professor Khateb and collaboratives about the dissociation between the semantic relation and language effects in the bilingual brain. It is well known that behavioral studies report semantic priming effects as faster responses to target words preceded by semantically related primes in comparison to semantically unrelated ones. In bilinguals, this effect is observed together with a language effect, which is faster in responses to targets in the first than in the second language. They have investigated the event-related potential correlates of the semantic and language effects in bilinguals, concluding the dissociation between language and semantic relation effects in time and space.

For the author of this chronicle, the Congress finished with a session about History of Neurology. Professor Iniesta spoke about History of Medicine in Spain and the figure of Pedro Laín Entralgo. In a very beautiful presentation, Professor De Felipe stressed the contribution of Santiago Ramón y Cajal to the study of the cerebral cortex. One of Cajal´s favourite topics was the study of the cerebral cortex and, especially, "the butterflies of the soul", the term he adopted as a metaphor for the pyramidal cells. Dendritic spines were first described by Cajal in 1888. Two years later, he also described the presence of spines on pyramidal neurond in the cerebral cortex of mammals, suggesting that these structures were points of contact with axon terminals. Others were sceptical and considered the spines as artifacts produced by the Golgi method. In fact, the postsynaptic nature of spines could not be demonstrated until the advent of electron microscopy in the 1950s. Many recent studies have shown that the dendritic spines of pyramidal neurons are highly plastic structures that appear to be very important elements in cognition.

The session was closed with a presentation of a video by Professor Alberto Portera. Professor Portera emphasized a fundamental question for this blog: the correlation between neuronal functioning and social coevolution.

This Congress has been made possible through the generous contribution of many people. We acknowledge, especially, the organizing effort of the Professors Antonio Gil-Nagel (Hospital Rúber Internacional-Madrid) and Jesús Porta-Etessam (Hospital Clínico-Madrid).

Implementation Intentions and Artificial Intelligence

In this article I will explain briefly the main conclusions presented by the author of this blog in Berlin ("International Congress of Psychology", 2008). My contribution, in collaboration with Professor Dr. Javier González Marqués (Chair of the Department of Basic Psychology at the Complutense University of Madrid), was entitled "Implementation Intentions and Artificial Agents" and establishes an interesting connection between social cognition in humans employing a particular type of intentions and its simulation and performance by intelligent artificial agents.

An intention is a type of mental state that regulates the transformation of motivational processes in volitional processes. Peter Gollwitzer distinguishes between goal intentions and implementation intentions. Goal intentions act in the strategic level whereas implementation intentions operate in the level of the planning. Goal intentions admit to be formulated by means of the expression "Intend to achieve X!", where X specifies a wished final state. However, implementation intentions can be enunciated like "I intend to do X when situation Y is encountered". This means that in an implementation intention, an anticipated situation or situational cue is linked to a certain goal directed behavior.

We have made a computer simulation that allows to compare the behavior of two artificial agents: both simulate the fulfillment of implementation intentions, but whereas one of them incarnates to A0 agent whose overturned behavior will be something more balanced towards the goal intention to obtain the reward R, A1 agent will reflect a more planning behavior, that is, more oriented towards the avoidance of obstacles and the advantage of the situational cues.

The hypothesis to demonstrate will consist of which, with a slight difference in the programming of both agents, A1 agent not only will yield a superior global performance but that will reach goal R before A0 in a greater number of occasions. This is clearly in consonance with the results of Gollwitzer and collaborators about the superiority to plan in humans the actions by means of implementation intentions as opposed to the mere attempt to execute a goal intention.

Gollwitzer and Sheeran (2004) have made a meta-analytical study of the effects exerted by the formulation of implementation intentions in the behavior of achievement of goals on the part of the agents. We set out to transfer the fundamental parameters with humans to A1 agent and to compare results with A0 agent more oriented to the execution of the goal intention to reach R. According to authors (2004, p. 26), the general impact of implementation intentions on the achievement of goals is of d=0.65, based on k=94 tests that implied 8461 participants. An important effect (op. cit., p. 29) was obtained for implementation intentions when the achievement of goals was blocked by adverse contextual influences (d=0.93). The accessibility to the situational cues was of d=0.95. To A1 we have assigned a 65 percent of percentage in the achievement of the goal. We have located a difference of 30 points in the achievement of R and according to a difference of percentage in the achievement of R on the part of A0 of 16 points, A0 was assigned a degree of achievement of 81 percent. As for the accessibility of the situational cues L, this one is very high in A1 agent (95) and considering that A1 can add 30 points more than A0, taking advantage of the situations, we have assigned to A0 a percentage of accessibility of the 76 percent. Considering that the degree of avoidance of obstacles S on the part of A1 is very high (93), to A0, we have assigned a difference to it of 19 points, that is to say, of the 74 percent. However, to fall in anyone of places S counts equal reason why it affects to the penalty for both agents.

We give account of the results, once made 5000 trials, with an average of about 48 movements by trial. We have considered the total number of plays, points (average), total resumptions (average), total victories or the number of times that the agent reaches R in the first place, number of situational cues L, number of obstacles S and number of carried out movements. The system of assigned points was:

A0: start: +50; L0-L5: +20; S0-S5: -5; R: +150; D0 (dissuasive agent intercepting the agents A0 and A1): -150; penalty by each movement: -1.

A1: start: +50; L0-L5: +25: S0-S5: -5; R: +120; D0 (dissuasive agent intercepting the agents A0 and A1): -150; penalty by each movement: -1.

The diversity of tasks that the agents have to execute in the board ends up interacting of a dynamic and significant way. This still is appraised with greater forcefulness in the one perhaps that it is the most decisive and surprising result of this exercise of simulation: the one that the most planning agent A1 achieves goal R in a greater percentage of times than A0, when A0 has been programmed to perceive and to accede to R with greater facility. We believe that our simulation has fulfilled the basic objective of supporting, in the area of Artificial Intelligence, the experimental conclusions with humans, of Gollwitzer and other authors about the superiority of the use of implementation intentions in the goal achievement, against the emphasis located in the execution of the goal intentions. As an obvious result, this task, given its limited nature, has not collected all the possibilities. Thus, the issue of the beginning of goal purpose has not been approached. Neither has the issue of the fact that the agents abandon the purpose of reaching R or that they seek alternative goals. On the other hand, not even the effect on the learning of the task as consequence of successive frustrations has been outlined. It would be interesting, to introduce agents not only based on learning rules but also adaptive agents.