Imitation, Maslow´s Pyramid and Multi-Agent Systems

We review an interesting article published by Le Guen Herve and Moga Sorin in Proceedings of International Joint Conference on Neural Networks (2009).

The authors study the influence of imitation within a population of artificial agents following Maslow´s Pyramid of needs. Imitation is the possible origin of communication and social learning. The imitative abilities include many types like facial imitation, perception by the infant that he or she is being imitated and empathy. The interest in empathy induces the study of emotions which Herve and Sorin link to the analysis of social needs as modeled by the Maslow´s Pyramid of motivations. Although the influence of each particular need varies from one person to another the principle is there are two main classes of needs. The most important needs are called primary needs (eating, breathing, etc.) while the second class of needs constitute the social aspect of human motivation and includes the need for esteem. This need is bidirectional and has been interpreted by the authors as the need to imitate and the need to be imitated. As the fact of being imitated is perceived by the agent concerned and as imitation is likely to generate empathic satisfaction, the authors model them as expected rewards. Based on a model of an autonomous robot with goal-oriented navigation and imitation capabilities, the robot goals are derived from internal variables that have to be maintained in a comfort area. The values of these variables decrease in time. The robot population´s task is to explore an unknown environment and to localize sources corresponding to its needs. Its survival will depend upon the satisfaction of these needs by discovering the different locations of the sources. The behaviors are obstacle avoidance, goal-oriented navigation, imitation and exploration. The emotional signature expresses the current state and the expected state of the agent: an agent in its comfort area displays a neutral signature whereas an internal variable below a certain threshold induces pain. That pain will cause a potential empathic response that is likely to incite another agent to move toward a known source. Besides a motivated agent is likely to provoke an attractive empathic response: keeping a motivated agent in its own field of perception becomes a source of motivation. It leads to the selection of the imitation target according to its motivational state.

The results of the simulation with Maslow agents showed an enhancement of the global survival rate even with a very small population wherein communications are not frequent.

The authors have proposed a holistic approach to the implementation of imitation in autonomous agents.

We recommend this excellent article because its ideas permit to build a simple and scalable model of agent useful in applications such as networks or swarm piloting.

Agent Societies and Artificial Intelligence

Very recent articles have addressed the problem of the interaction between populations of virtual robots. Vijayakumar and Davis ("Metacognition in a "society of mind"") investigate the concept of mind as a control system using the "Society of Mind" idea from Marvin Minsky. They develop Metacognition in a model based on the differentiation between metacognitive strategies. Metacontrol is a part of metacognition. The authors explore Metacognition mechanisms in developing optimal agents for the fungus world testbed. The fungus world environment allows the behaviours of a robot to be monitored, measured and compared. It is generated a swarm intelligence of how the group of agents work together to achieve a common goal. Vijayakumar and Davis implement an architecture with six layers including reflexive, reactive, deliberative, learning, metacontrol and metacognition layers. The authors design an experiment with different types of agents (random, reflexive, reactive...) All agents move in the environment, changing direction in case of obstacles. To compare the results of each agent, the following data were collected: life expectancy, fungus consumption, resource collection and metabolism. Total performance was denoted by the combination of resource collected and life expectancy. Experiments were conducted for each type of agent. In Simulation 1, deliberative agents collected 50% of resource and left 64% of energy. In Simulation 2, the results of deliberative agents were compared to Metacognition agents, collecting metacognitive agents 82% of resource and lefting 71% of energy. The result concludes that Metacognition agents are better than other cognition and deliberative agents. Thus, Metacognition is a very powerful tool for control and self-reflection and intelligent and optimal agents can be viewed as collective behaviours as a "Society of Mind". Really to develop a Metacognition concept in Artificial Intelligence seems necessary for building self configurable computational models and true intelligence.

MathPsych 2009 (Amsterdam)

The Annual Convention of the Society for Mathematical Psychology ("MathPsych 2009") was held in Amsterdam on August 1-4, 2009. We stress the highlights for the purpose of our blog, Social Cognition.

On August 2, Joe Johnson (Miami University) illustrated, via a simple mathematical model, the ability for predicting decision behavior based solely on perceptual data. Johnson uses an evidence accumulation model with a ratio choice rule to predict athletes´intuitive, initial choice in a realistic game situation. Markus Raab (German Sport University in Cologne) investigates the preference for intuitive decisions in contrast to deliberative ones. He applies a mathematical choice model based in team handball attack situations. Bayesian models also are applied and so Tom Lodewyckx (University of Leuven) and others design a Bayesian state-space model for affectivity. They use Markov chain Monte Carlo methods to estimate the model parameters. This framework is used to high resolution psychophysiological and behavioral data obtained during adolescent-parent interactions expressing dynamical emotions.

Vanpaemel (Umiversity of Leuven) and Michael Lee (University of California, Irvine) advocate the advantages of hierarchical Bayesian modeling in providing one way of specifying theoretically-based priors for competing models of category learning. In a similar line, the group of Rich Shiffrin (Indiana University) spoke about information integration in perceptual decision making. According to the authors, researchers studying judgment and decision making have shown that people employ sub-optimal strategies when integrating information fron multiple sources. But another group of researchers has had success using Bayesian optimal models to explain information integration in fields such as perception, memory and categorization. Shiffrin and colaborators design a decision making experiment to test the range of this difference.

Amy Perfors and Daniel Navarro (University of Adelaide) consider the situation in which a reasoner must induce the rule that explains an observed sequence of data, but the hypothesis space of possible rules is not explicitly enumerated or identified. They present mathematical optimality results showing that as long the hypotheses tend to be sparse (that is, tend to be true only for a small proportion of entities in the world), then confirmation bias is a near-optimal strategy. The authors propose, in a very interesting manner, to chose queries that one knows will lead to an affirmative response for at least some hypotheses (hypotheses being considered). This positive-test strategy is closely related to the confirmation bias.

The Meeting finished on August 4 hoping to achieve the same successful objectives for coming events.

Public Good Games and Social Cognition

Public good games provide an interesting testbed for the study of social dilemmas. Social dilemmas are inherent in decision making. Despite the diversity of dilemmas, all share the same structure: each individual benefits from behaving selfishly but a group gets greater rewards if its members cooperate.

In a typical public good game, players may divide their initial endowments into both a private account and a group account. The frequent strategy for a rational player is defection but most participants in public good experiments do cooperate to some extent. Factors that enhance cooperation are the communication among players or the payoff structure of the game. Nowadays it is thought that behavioral investments could be an important determinant of cooperation in social dilemmas. Participants´ endowments are distributed asymmetrically among a group to discover whether "rich" individuals will contribute more than poor individuals. The conclusion is that the differences in the level of contributions may be reduced if positions are assigned on the basis of merit rather than chance. Although asymmetries in participants´ behavioral investments (for instance, effort investments) affect fairness judgments and group identification, members are often not aware of such differences. But the origin of the endowments affects contribution levels in the public good game (Muehlbacher and Kirchler, 2009). Subjects who earned their endowments through a greater amount of effort were less cooperative than those individuals who had earned the money with ease.

Chaos Theory and Social Cognition

We will review the last contributions of the theory of nonlinear dynamical systems to the field of social cognition, specially

Vallacher and Nowak (2009).

Social relations evolve and change in the absence of external influences. Psychological systems display intrinsic dynamics. Three basic types of attractors have been identified: fixed-point attractors, periodic attractors and deterministic chaos. A fixed-point attractor describes the case in which the the state of the system converges to a stable value. It is similar to the notion of homeostasis. It corresponds, for example, to a desired goal. Multiple fixed-point attractors express that people can have different (perhaps contradictory) goals and patterns of social behavior.

Some systems display oscillatory behavior. A temporal pattern showing this tendency is a periodic attractor. Social judgement often oscillates between positive and negative assessments.

Chaos represents a possibility in many social phenomena. Social psychology presents many nonlinear phenomena, such as complex interactions among variables or inverted-U relations.

Social interdependence is very important to game theoretic approaches in Psychology, for instance, the prisoner´s dilemma game. So, Nowak et al. (1990) used cellular automata to model the dynamics of social influence. Each individual had three properties: an opinion on a topic, a degree of persuasive strength, ans a position in a social space. In each round of the computer simulations, one individual was chosen and influence was computed for each opinion in the group. The updating rule was the following: the individual changed the opinion to match the prevailing opinion if the resultant strength for this opinion position was greater than the strength of the individual´s current position. The minority opinion survived by forming clusters of like-minded people.

The dynamical account of social influence describes how the state of a single individual depends on the state of other individuals. However, individuals are best conceptualized as displaying patterns of change rather as a set of states. Social influence can be approached as the coordination over time of individual dynamics. Individuals in a relationship are represented as separate systems capable of displaying rich dynamics. A recent model of synchronization has been developed by Nowak, Vallacher and Zochowski (2002). Coupled logistic maps are used in this model. The behavior of each individual not only depends on the preceding state but also on the preceding state of the other person.

Dynamical social psychology has generated a deep source of formalisms but social reality can not be confused with physical reality. Individuals are not interchangeable in the way that atoms are. People live in a symbolic world and do not respond in a reactive way to the objective features if the environment. For it, human dynamics contain some degree of randomness, and human behavior is often unpredictable and perhaps chaotic.

Clint Sprott (Fractal image)

Peter Gollwitzer and social cognition

I would like to pay homage to Peter Gollwitzer, presenting some of his outstanding contributions to social cognition.

Peter Gollwitzer represents the great German tradition in Psychology: from Wilhelm Wundt to Kurt Lewin, from Asch to Heinz Heckhausen, Germany always has brought excellent researchers to the study of the volitional processes. Gollwitzer is renamed expecially by his contributions to intentional behavior studying implementation intentions. Intention implementations are currently at the center of research in the world of the Psychology. By means of implementation intentions, bringing a goal pursuit to a successful end is facilitated. They are effectives in different areas such as health domain or academic performance. So, the likelihood of performing a breast revision was enhanced by forming implementation intentions. Besides, the effects of implementation intentions on reducing dietary fat intake were relevant. Bad habits are controled by implementation intentions and Schweiger Gallo has verified the effectiveness of forming implementation intentions on the suppression of undesired emotional responses (for instance, fear of spiders). But perhaps more important is the study of implementation intentions effects on critical populations, such as addicts in withdrawal, schizophrenic patients or subjects with frontal brain lesions. Another clinical population who profits from implementation intentions is ADHD children: in a study, the ADHD-children who formed an implementation intention showed the same response inhibition performance as children without any psychological disorders.

But perhaps the two pending tasks is to study the design of Artificial Intelligent systems incorporating implementation intentions and the underlying mechanisms of implementation due to neurophysiological research. The first one has became to be analysed by the author of this blog in his article about implementation intentions and artificial agents. The second one is being studied by means of the electrocortical correlates of emotion regulation by ignore-implementation intentions. It seems that these intentions produce their effects through cortical control that sets in the information processing system blocking the emergence of negative emotions. Electrocortical correlates offer the possibility of determining at what point different kinds of implementation intentions (suppression or inhibition) exert their effects after the critical stimuli are encountered.

Forming implementation intentions reveals to be an effective self-regulatory instrument in domains including social psychology and clinical psychology. Gollwitzer continues the tradition.

Joint Action and Artificial Intelligence

In this article, we examine how the research about joint action (Sebanz, Knoblich, Bekkering...), can be intertwisted to the ideas about distributed coordination in uncertain multiagent systems (Maheswaran, Szekely, Rogers, Sánchez...)."Joint action can be regarded as any form of social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment" (Sebanz, Bekkering, Knoblich, 2006, p. 70). According to the authors, successful joint action depends on the abilities to share representations, to predict actions and to integrate predicted effects of own and other´s actions. A mechanism for sharing representations of objects and events is to direct one´s attention to where an interaction partner is attending. However, a more direct mechanism is provided by action observation; studies about mirror neurons show that during observation of an action, a corresponding representation in the observer´s action system is activated. On the other hand, an efficient means to predict other´s actions that is not based on action observation is knowing what another´s task is. A series of recent studies has shown that individuals form shared representations of tasks quasiautomatically, even when it is more effective to ignore one another. But how individuals adjust their actions to those of another person in time and space can not be explained just by the assumption that representations are shared; action coordination is achieved by integrating the "what" and "when" of others´actions in one´s own action planning. This affects the perception of object affordances, and permits joint anticipatory action control. Very recently, Maheswaran, Rogers and Sanchez (2007) introduce the idea of distributed coordination in multiagent systems. I believe that the research by Sebanz and others in humans can be applied successfully to the area of distributed Artificial Intelligence. Centralized systems can generate fully coordinated policies but put a very high computational load on a single agent. Given a team of agents, every agent in the team has a set of activities that it can perform. Each activity has probabilistic outcomes. Only the agent has current knowledge of its policy at all times. The team reward is a function of the qualities of all activities, and the agents´ objective is to maximize this reward at some terminal time. One way this function can be composed is with a tree where the activities are leaf nodes. Each non-leaf node is associated with an "ancestral" operator which takes the qualities of its children as input. The output of the root node is the team reward function. An agent´s subjective view of the reward function can be defined. It is considered the case where each agent see all ancestral nodes of activities they own, and any nodes and links that connect to its activities and their ancestral nodes via directional operators. The authors introduce distributed coordination between artificial agents, that is, something like the joint action in humans studied by Sebanz and others in humans. In an immediate future, a fertile cross will happen between these two roads.

Winter School 2009: Social Structures in Communication Networks

On 5-10 January 2009 and chaired by Jorge Louça, has been celebrated at the University of Lisboa, the Winter School 2009: "Social Structures in Communication Networks". On Monday, Professor Louça presented the program and scientific goals in the context of the International Doctoral Program in Complexity Sciences. The "Universidade" is accomplishing a great effort following the way pioneered by Santa Fe Institute or the Complex Science Society at Paris.

Professor Araújo analysed whether biological-inspired network models contribute to the understanding of aggregate economic behaviors. Besides, and following the ideas by John von Neumann, she explained the redundant nature rather than efficient nature of the network structures, contributing to clarify the idea of efficiency versus reliability in social environments. In the evening, Professors Louça and Rodrigues introduced tools for the study of social networks like NWB, PAJEK, GUESS or UCINET.

On Tuesday, Professor Symons introduced in his second presentation, ideas including references to the analysis of emergent properties. Symons offers an alternative to the macro-properties model of emergence. Emergence is not restricted to macro-phenomena, but can appear at the intersection of networks. Objects or agents can be involved in distinguishable systems or networks simultaneously. An agent may participate in social, economic, political and other networks at the same time.

On Wednesday, Professors Louça and Rodrigues showed multi-agent based social simulation by means of tools like NETLOGO, MASON or BREVE, while on Thursday, both Professors outlined community detection in social networks, employing algorithms like Girvan-Newmann. In the evening, Professor Palla exposed the statistical properties of community evolution in complex networks. On Friday, Professor Marinheiro spoke about network architectures and organization. Network maps have been used to mitigate the problem of the understanding of the network structures. Professor Lopes explained that in the networked multimedia realm, "more is different".

Finally, on Saturday, 10th, it was programmed a keynote talk by James Sterbenz (University of Kansas), concerning adaptive computer network architectures.

I am very grateful to the University of Lisboa by the organization of this School in which young students and prestigious Professors, have shared very interesting viewpoints about the formal and social analysis of networks, mixing complex systems and dynamics of networks.

"The brain is a network of neurons; organizations are people networks; the global economy is a network of national economies, which are networks of markets...How do such networks matter?"

The link to the Winter School is:

http://idpcc.dcti.iscte.pt/ws2009/ws2009_home.html

Synergetics and social cognition

The term "synergetics" was introduced by Hermann Haken, pioneer in the study of the laser, on 1970. It means the science of cooperation and can be considered as a science of orderly, self-organized, collective behavior subject to general laws. Therefore, the aim of synergetics is to establish the natural laws on which the self-organization of systems is based.

In physics there are different aggregate states-solid, liquid, gaseous-called phases, and the transitions between them are called phase transitions. The three phases differ only in the arrangement of the molecules. If we heat a layer of liquid in a dish from below and if the temperature difference berween top and bottom is only slight, there will be no motion of the liquid on a macrolevel. But when the temperature difference is further increased the liquid begins to move macroscopically in a quite orderly manner in the form of rolls. The curious fact is that such hot drops do not rise irregularly but in an orderly manner. Nature discovers that it can transport the heated parts upward more efficiently when they join in a regular motion. If we add the individual motions of the rolls, we obtain a hexagonal pattern. The liquid rises in the center of the hexagons and sinks along the outside:

Once the choice is made the alternatives are out of the question, and the choice cannot be reversed. Minor fluctuations decide the nature of the choice. Once it has been made, all particles must accept it. Increasingly complex motion patterns can be created by self-organization, that is, and employing the language of synergetics, new order parameters succeed each other.

Nowadays, many concepts of the synergetics, like "attractor", "bifurcation", "fluctuation", "synchronization effect", "symmetry broken"... are useful for the application to social sciences. For instance, to social conflicts. According to Haken, conflicts exist that offer two equivalent solutions in which society resolves the conflict for individuals merely displacing it. Do the courts favor the mother or the father in the child´s upbringing? (Haken, 1984). The symmetry must be broken by the judge. The advantages and disadvantages of one solution are balanced against those of the other.

Neuro-psychoanalysis: a new revolution?

In this brief article we will explain the current connections between psychoanalytic concepts, like Ego or Id, and Neurology and the computational study of the episodic memory for artificial agents. We will refer to the book by Solms and Turnbull, "The Brain and the Inner World" and to the recent work by Andreas Gruber at the Vienna University.

In 1923, Freud recognized that the rational part of the mind is not necessarily conscious: consciousness was not a fundamental organizing principle of the architecture of the mind. Freud attributed the functional properties previously assigned to the system Conscious-Preconscious to the "Ego", being conscious only a small portion of the Ego´s activities. Its main property was the capacity for inhibition. Freud thought of the capacity to inhibit drive energies, the basis of all the Ego´s rational. Consciousness over time allows the development of what Damasio (1999) calls an "autobiographical self". In psychoanalytic terms, the core "self" might be described as a perception of the current state of the "Id", whereas the autobiographical self is synonymous with the "Ego". Episodic memory is the fundamental element of the "autobiographical self". Following to Schacter (1996), the episodic memory system allows explicitly to recall the personal incidents that define the lives of human beings. Very recently, Andreas Gruber (2007) at the Vienna University, has built a computational model of episodic memory that is based on Freud´s Ego-Superego-Id personality model. A pre-decision module, representing the Freudian "Id", consists of the drives and the basic emotions modules. In this model, low-level decision making is done. If a drive is very high, the system will try to bring the drive back into its balanced range. By the other side, perceived situations are handed over to the the high-level decision making module: it represents the "Ego". Decision module interacts with the episodic memory by searching for similar situations to the current one including their emotional rating. A behavior may be triggered directly by a complex emotion while reactive responses arise from the pre-decision module, whereas routines are longer sequences of actions stored in the procedural memory.

It is very curious to verify that the Freudian paradigm, always impugned by the absence of scientific methodological rigour, currently is guiding the search in Neurology and Artificial Intelligence. Are we witnessing a new and revolutionary synthesis?

Dunlosky & Metcalfe: "Metacognition"

Two prestigious authors, John Dunlosky and Janet Metcalfe, have published on September the first introduction to Metacognition for undergraduates. Dunlosky is a Professor of Psychology at Kent State University and Metcalfe is a Professor of Psychology and of Neurobiology and Behavior at Columbia University. This book is an excellent tool for understanding in a concise way the state of the art of the emergent field of the Metacognition. From a didactic point of view, we remark the presence in each chapter of boxes presenting specific subjects and closing the chapters, questions discussion and review concepts which allow to consolidate the learning.

An introduction starts presenting the metacognitive model of Nelson and Narens (1990), the most followed model in this area. In this model, the interplay between the meta-levels and the object-level defines the two process-based activities of metacognition: monitoring and control: control is exerted whenever the meta-level modifies the object-level; controlling the object-level provides no information about the states of the object-level. For it, you must monitor the object-level activities so that you can update the model of them.

Chapter 2 presents a very interesting History about the theoretical development of the concept. Perhaps it´s the first historical compendium published. Since the modern pioneer John Flavell to Asher Koriat many names and photos of researchers are included.

Section 1 analyzes the main empirical methods for investigating the Metacognition and chapter 4 speaks about feeling-of-Knowing (FOK) judgments and Tip-of-the-Tongue states. In the first case, Dunlosky and Metcalfe introduce theories like "Target Stregth" (Hart, 1967), "Cue Familiarity" (Reder, 1987) or "Target Accessibility" (Koriat, 1993). An interesting section about the contributions by Lynne Reder and colleagues on strategy selection by means of (FOK) judgments, finishes the chapter.

Judgments of learning (JOLs) are decisive for Education. In a seminal work, Arbuckle and Cuddy (1969) argued that if paired associates "differed in associative strengths immediately following presentation, subjects should be able to detect these differences just as they can detect differences in strength of any other input signal". Their students studied short lists of paired associates, and inmediately after studying each one, they judged whether or not they would recall it. Following Dunlosky and Metcalfe (p. 94), research on JOLs has focused on contexts in which extra study typically does improve memory. Besides self-feedback from intervening test trials does benefit the relative accuracy of JOLs. There are several hypothesis explaning the effects of JOLs. Begg et al. (1989) proposed that JOLs are based on the ease of processing an item inmediately prior to making the judgment. Bjork and Schwartz (1998) have demonstrated the effect of retrieval fluency on JOLs. Koriat (1997) has used the cue-utilization approach. Acording to this perspective, people use a variety of cues-item relatedness, number of study trials...-to infer whether an item will be remembered.

Confidence judgments or retrospective confidence (RC) judgments are analysed in chapter six. What causes the overconfidence effect? Perhaps cognitive biases distort people´s judgments but the overconfidence effect is probably an artifact of experimental methods.

Source judgments (chapter seven) involve remembering the source of a memory or the context in which a memory originally occurred. Between the factors that influence source-monitoring accuracy are the similarity of the sources and emotion and imagery.

Chapter eight is a very interesting one. The authors review some findings on people´s assessment of the truth of witnesses´memories based on their expressed confidence, and on people´s abilities to detect lies. Many cases are presented specially about hindsight bias: this effect has important implications for the criminal justice system, for isues as whether jurors are able to disregard testimony that has been ruled inadmisible.

In chapter nine, are considered in detail the wide connections of Metacognition to Education and this textbook finishes including a section about life-span development. We remark in chapter ten the interesting references on development of theory of mind and metacognition. In the other side (chapter eleven), it´s reviewed the problem of aging and memory monitoring.

This book is not only the first textbook to focus on Metacognition but perhaps the first book in covering metacognitive research in an unified framework. It´s also an excellent handbook for more advanced students and has been writen by two authentic champions of the subject.

Martin Nowak and Cooperation

Martin A. Nowak, the master of the spatial games published two years ago a book, entitled "Evolutionary Dynamics: Exploring the Equations of Life" (Belknap Press, 2006). We remark some references by Nowak about cooperation in this book.

According to Nowak, experimental game theory shows that humans do not behave rationally. They are guided by instincts that might have evolved via different situations. In the Prisoner´s Dilemma, humans often try to cooperate. Cooperation means not to cooperate with the state attorney but to cooperate with your partner and remain silent. If both of you defect, both will get a long prison sentence. No matter what your partner does, it is better for you to defect. The rational analysis suggests that both prisoners will confess and spend a long time in jail. Defector always have a higher fitness than cooperators. Natural selection increases the frequency of defectors until cooperators have become extinct. But cooperation becomes a promising option when the game is repeated several times between the same two players. Consider a strategy in which an agent cooperates on the first move and then cooperates as long as the opponent does not defect. If the opponent defects once it will switch permanently to defection and it will never forgive. This strategy is a strict Nash equilibrium versus a strategy consisting in always defect. In terms of evolutionary dynamics, if the whole population uses the first strategy, then the second one cannot invade. Obviously, human strategic instincts are not formed by playing games with a well-defined number of rounds. In the lifetime there might always be another round. There is always a tomorrow in our plans.

Towards a computational model of craving in addictions

Several computational models try to explain the brain mechanisms in the origin of addictive behaviors. Perhaps, David A. Redish, Professor at the Department of Neuroscience (Minnesota University) has been the pioneer. He published in Science (2004) the article "Addiction as a computational process gone awry", a milestone for many other computational models.

Following to Redish and Johnson (2007), there are two systems in the mammalian brain with differing levels of search: (1) a flexible system, which is capable of being learned quickly, but is computationally expensive to use, and (2) an inflexible system, which can act quickly, but must be learned slowly. The flexible system allows the planning of multiple paths to achieve a goal; the inflexible system only retrieves the remembered action for a given situation. The authors hypothesize a unified system incorporating three subsystems, a situation recognition system and two contrasting decision systems-a flexible, planning-capable system that accommodates multiple paths to goals and takes into account the value of potential outcomes, and an inflexible, habit-like system.

The flexible decision-making system requires recognition of a situation, recognition of a means of achieving outcome from situation as well as the evaluation of the value of achieving outcome.

The inflexible decision-making system entails a simple association between situation and action. Evaluation entails a memory recall of the learned associated value of taking an action in the situation. To evaluate the value of an outcome, the system needs a signal that recognizes hedonic value. Two brain structures that have been suggested to be involved in the evaluation of an outcome are the orbitofrontal cortex and the ventral striatum. Neurons in the ventral striatum show reward correlates, and anticipate predicted reward. The hippocampus projects to ventral striatum and ventral striatal firing patterns reflect hippocampal activity.

It seems that hedonic signals are carried by opioid signaling. If endogenous opioids signal the actual hedonic evaluation of an achieved outcome, then when faced with potential outcome signals arriving from the hippocampus, one might expect similar processes to evaluate the value of expected outcomes. According to Redish and Johnson (art. cit., p. 330), this predicts that the effect of hippocampal planning signals on accumbens structures will be to trigger evaluative processes similar to those that occur in response to actual achieved outcomes. This has an effect for craving. Craving can be defined as the intense desire for something. Because the flexible system only entails the recognition that an action can lead to a potential path to a goal and does not entail a commitment to action, craving is not necessarily going to produce action selection. When the hippocampal component reaches a goal that is evaluated to have a high value, this will produce a strong desire to achieve that goal: the psychological effect of that recognition is to produce craving.

Competitive opioid antagonists have been used clinically to reduce craving. The hypothesis that reward signals are released on recognition of a pathway to a high-value outcome implies that blocking those reward signals would not only reduce the subjective hedonic value of receiving reward, but would also reduce craving for those rewards. If that reward signal is based on opioid signaling, then this may explain why an opioid antagonist such as naltrexone can reduce craving.

José Mira: in memoriam

On August 13th, 2008, Spanish engineer José Mira died in Madrid. Specialists in Artificial Intelligence and others will remember Mira as a sharp thinker and a nice person, whose interests and expertise ranged from theoretical and applied Artificial Intelligence to Cognitive Science. Professor Mira did much to establish in 1989 the Department of Artificial Intelligence at UNED. Mira' s research contributions were focused on methodological aspects in Knowledge Engineering and the connections between Neuroscience and Computation. From an applied perspective he worked about Knowledge Systems in Medicine and Industry. Definitely, Mira has been one of the principal influences on AI in Spain and Latin America: his death feels like losing a decisive influence.

"Fungus Eaters" and Artificial Intelligence: a tribute to Masanao Toda

In 1962 Masanao Toda published a revolutionary study that was the origin of the idea of virtual complete autonomous agents and that preluded branches very important in Artificial Intelligence like autonomous robots (Brooks, 1991), swarm intelligence (Bonabeau, Dorigo and Theraulaz, 1999) or evolutionary robotics (Nolfi and Floreano, 2001). Against a cognitive science centered in a symbolic perspective, Toda analyses systems capable of behaving autonomously in an environment without external aids. This is a very hard problem in Artificial Intelligence. Complete autonomous systems have to incorporate capabilities for deciding what to do interacting with the environment.

In his article "The design of a fungus eater" (Behavioral Science 7, 164-83), Toda "consider(s) a design of a robot to be sent to a hypothetical planet as a robot uranium miner, which sustains itself by eating fungi as its energy source". This solitary creature feeds on a type of fungus that grows on the planet and the more uranium ore collects, the more reward it will get (see also Pfeifer, 1996, 3-12). The "Solitary Fungus Eater" (SFE) has means of collection and locomotion and means for decision making interacting with the perception of the environment. It must be autonomous and self-sufficient because it can not receive external aids: it is a virtual adaptive system, a genial idea in 1961-62. Besides Toda introduces another insight: the notion of bodily designed agent. He employs a choice program for regulating the behavior of the "Eater" using stochastic distributions. Toda defines the expected amount of uranium exploitation by means of a function f indicating the expected amount of uranium exploitation as a function of fungus storage (Toda, 1982, p. 114). The problem is that f cannot be determined until the choice program is fixed and the optimal choice program is given only after the function f is specified. The key is in defining adaptability in an unknown world.

But Toda advances almost 30 years and proposes for investigating matters like emotion, irrationality, social psychology, etc., for autonomous agents. Does SFE have emotion? Of course, the answer depends upon the definition and perhaps it can be defined as a particular state of mind accompanied by a high level of energy mobilization. After Rosalin Picard will introduce the paradigm of "affective computing" and nowadays many researchers design virtual agents or robots incorporating emotions or something like that: Lola Cañamero, Eva Hudlicka, Darryl Davis, Rolf Pfeifer, Thomas Wehrle and many others (see the excellent introduction by Ruebenstrunk, 1998, http://www.ruebenstrunk.de/)

Besides Toda thinks that apparently irrational behavior of human being can also be incorporated in the SFE. Herbert Simon had spoke about bounded rationality, notion exploited after by Stuart Russell or Gerd Gigerenzer and, in the meantime, the omnipresent contribution by Toda.

And what to say about the communication and cooperation between agents? Toda (1982, p. 127) believes that "optimal cooperation programs for Fungus Eaters will provide a heuristic for the sciences of human interaction" and...for robotics? Chains of e-pucks robots colaborating between them (see Francesco Mondada in http://www.e-puck.org/) demonstrate this prophecy in 2008.

Masanao Toda passed away on 5th September 2006. Who was Masanao Toda? Was he the prophet of a new world inhabited by artificial autonomous agents?

Social Cognition in Berlin: "XXIX International Congress of Psychology"

The "XXIX International Congress of Psychology", organized by "The German Federation of Psychologists´Associations", under the auspices of the "International Union of Psychological Science (IUPsyS)" was held in Berlin on July 20-25, 2008.

On Monday, 21 July, Don Spangler in "Computer measurement of motives", described the development and validation of a computer program to measure implicit motives. This program analyses namely speeches, written materials, interviews, transcripts of meetings, and articles appearing in the business press of USA.

Hugo Kehr gave an overview of his compensatory model of motivation and volition. Structural components of the model are implicit motives, explicit motives, and perceived abilities. Its functional processes are volitional regulation (compensating for inadequate motivation) and problem solving (compensating for inadequate perceived abilities).

In "Routines and decision making", Tilmann Betsch emphasizes that empirical evidence indicates that routines influence behavior generation, information search, appraisal, choice and the implementation of a chosen behavior in human decision makers.

Professor Gao presented a computer program which can divide the critical thinking procedure into several distinctive stages, and diagnose individual logic fallacy instantly.

In the "Social Cognition I" session, Julia Herfordt spoke about social facilitation. Professor Herfordt tested Zajonc´s social facilitation hypothesis with an antisaccade task.

Professors Shi and Wang argued that two different imputations in opposite directions may occur at the starting point of the empathy process and that whether the information of imputation comes from external or internal source will determine which one really happens: the results showed that the difference of information source affected the way how participants used the information to infer what other people would know and act.

To end this paper session, Professor López Alonso analyzed social representations from their inferential bases and structures as cognitive processes.

The paper session of "Social Psychology" was opened by an interesting contribution about the possible existence of structures isomorphic to feedback cycles in social psychological phenomena; Professor Caraiani analyzed classes of symmetries and synchronization of chaotic phenomena through the consideration of the correlation between cyclic invariants.

On Tuesday, 22 July, Professor Bertrams chaired a Symposium about "Self-regulatory strength and ego depletion". According to Professor Schmeichel, both cognitive load and ego depletion undermine self-control but they don´t operate via the same mechanism.

Professor Bertrams believes that success in self-control and complex thinking depends on a resource comparable to the strength of a muscle. This regulatory strength can be boosted by regular self-control effort. In the afternoon, Ulrich Wiesmann presented his structural modelling approach for the generalized health-related self-concept. Professor Wiesmann starts out from Markus´dynamic self-concept theory and finds support for a hierarchical factor structure.

On Wednesday, 23 July, a paper session about "Self-regulation" was chaired by Professors Sassenberg and Mamali. Jan Crusius analyzed the role of spontaneous social comparisons in automatic goal pursuit. Professor Crusius suggests that social comparison is a mechanism contributing to goal priming.

Waclaw Bak specified three dimensions of self-regulation. Based on conceptions of of Higgins, Markus, and Ogilvie, the self-system is defined as a cognitive structure, composed of different self beliefs (ideal-self, ought-self, undesired-selves, can-self, impossible-self) and discrepancies between them. Professor Bak hypothesizes that the self-system one can describe in terms of three dimensions: (a) negative-self standards; (b) positive self-standards and (c) can-self-standards.

The paper session "Social Cognition II", chaired by Carlos Pelta, was opened by Professor Kwong with an oral presentation on perceptions of progress towards goal. Following to Professor Kwong, perceived progress plays a critical role in people´s motivation to persist toward a goal. She has been demonstrated that while figure displays exert dominant influences over corresponding numerical displays, this effect is dependent on the relative fluency in processing the two different modes of information.

Professor Pelta presented "Implementation intentions and artificial agents" in collaboration with Professor González Marqués at the Complutense University of Madrid. The contents of this presentation have been exposed thoroughly in this blog.

Finally, in the paper session about the impact of goals and volitional processes on learning, Katrin Jorke presented Trait Procrastination as a failure in self-regulation that can have a negative impact on the efficiency of self-regulated learning.

During the Congress, an exhibition of posters allowed to obtain relevant information about the progress of Psychology in the world. Special thanks to the Organization Committee for their professional contribution in organizing this manifestation. I hope that the XXX Congress in Cape Town will be a success.

12th Congress of the European Federation of Neurological Societies

The 12th Congress of the European Federation of Neurological Societies was held in Madrid on August 23-26, 2008. Approximately 1,800 abstracts have been submitted and more than 4,000 persons have participated. Seventeen satellite symposia were available covering the latest therapeutical and diagnostic acquisitions.

The Congress was opened by the President of the EFNS, Jacques L. De Reuck and the first course was entitled "How do I examine...?". Professors Gil-Nagel, Quinn and Clanet spoke about the diagnostic in epilepsy patients, in patients with suspected Parkinsonism and patients with a multiple sclerosis, respectively.

On Sunday 24, we emphasize a session about Neuro-estimulation for epilepsy. According to Professor Wadman, deep brain stimulation is a successful therapy for epilepsy patients. However, as it is not known what the underlying fundamental mechanism is, it is unclear where the best location for stimulation is. It is needed distinguish between acute intervention in ongoing, just started, or about to start epileptic seizure and an approach where the main goal is to chronically decrease excitability and so reduce the likelihood of the occurrence of a seizure. Another interesting aspect is the applicability of closed-loop strategies where the level of stimulation is controlled by the state of the brain. Professor Wadman thinks that in epilepsy there are reasons to assume that continuous stimulation is not necessarily the best approach.

On Sunday in the morning, Professor Barbro B. Johansson explained the fascinating matter of the brain plasticity and stroke rehabilitation. According to Professor Johansson, training in specific activities of daily living, and starting within the first week or weeks after stroke are important key factors in stroke rehabilitation. The healthy brain has a large capacity for automatic simultaneous processing and integration of sensory information. Cortical lesions interrupt networks that combine different regions, and the capacity for automatic processing of incoming stimuli is reduced. Cortical sensory and motor representation of the hand exerts inhibitory influences on the motor opposite cortex in healthy individuals. Based on the observation of an abnormally high inhibition from the intact side in patients with cerebral infarcts, has been hypothesized that this abnormality might adversely influence motor recovery.

Applicability of neuroimagen techniques has been a topic very important in this Congress. On Monday 25, Professor Berg presented imaging techniques of basal ganglia disorders. However, SPECT or PET techniques are limited because they do not discern between idiopathic Parkinson´s disease and atypical Parkinsonian syndrom. Special MRI techniques, like diffusion tensor imaging (DTI) and volumetry of the respective areas need to be applied.

On Tuesday 26, there was a session about Cognitive Neuropsychology. It was very interesting a presentation by Professor Khateb and collaboratives about the dissociation between the semantic relation and language effects in the bilingual brain. It is well known that behavioral studies report semantic priming effects as faster responses to target words preceded by semantically related primes in comparison to semantically unrelated ones. In bilinguals, this effect is observed together with a language effect, which is faster in responses to targets in the first than in the second language. They have investigated the event-related potential correlates of the semantic and language effects in bilinguals, concluding the dissociation between language and semantic relation effects in time and space.

For the author of this chronicle, the Congress finished with a session about History of Neurology. Professor Iniesta spoke about History of Medicine in Spain and the figure of Pedro Laín Entralgo. In a very beautiful presentation, Professor De Felipe stressed the contribution of Santiago Ramón y Cajal to the study of the cerebral cortex. One of Cajal´s favourite topics was the study of the cerebral cortex and, especially, "the butterflies of the soul", the term he adopted as a metaphor for the pyramidal cells. Dendritic spines were first described by Cajal in 1888. Two years later, he also described the presence of spines on pyramidal neurond in the cerebral cortex of mammals, suggesting that these structures were points of contact with axon terminals. Others were sceptical and considered the spines as artifacts produced by the Golgi method. In fact, the postsynaptic nature of spines could not be demonstrated until the advent of electron microscopy in the 1950s. Many recent studies have shown that the dendritic spines of pyramidal neurons are highly plastic structures that appear to be very important elements in cognition.

The session was closed with a presentation of a video by Professor Alberto Portera. Professor Portera emphasized a fundamental question for this blog: the correlation between neuronal functioning and social coevolution.

This Congress has been made possible through the generous contribution of many people. We acknowledge, especially, the organizing effort of the Professors Antonio Gil-Nagel (Hospital Rúber Internacional-Madrid) and Jesús Porta-Etessam (Hospital Clínico-Madrid).

Implementation Intentions and Artificial Intelligence

In this article I will explain briefly the main conclusions presented by the author of this blog in Berlin ("International Congress of Psychology", 2008). My contribution, in collaboration with Professor Dr. Javier González Marqués (Chair of the Department of Basic Psychology at the Complutense University of Madrid), was entitled "Implementation Intentions and Artificial Agents" and establishes an interesting connection between social cognition in humans employing a particular type of intentions and its simulation and performance by intelligent artificial agents.

An intention is a type of mental state that regulates the transformation of motivational processes in volitional processes. Peter Gollwitzer distinguishes between goal intentions and implementation intentions. Goal intentions act in the strategic level whereas implementation intentions operate in the level of the planning. Goal intentions admit to be formulated by means of the expression "Intend to achieve X!", where X specifies a wished final state. However, implementation intentions can be enunciated like "I intend to do X when situation Y is encountered". This means that in an implementation intention, an anticipated situation or situational cue is linked to a certain goal directed behavior.

We have made a computer simulation that allows to compare the behavior of two artificial agents: both simulate the fulfillment of implementation intentions, but whereas one of them incarnates to A0 agent whose overturned behavior will be something more balanced towards the goal intention to obtain the reward R, A1 agent will reflect a more planning behavior, that is, more oriented towards the avoidance of obstacles and the advantage of the situational cues.

The hypothesis to demonstrate will consist of which, with a slight difference in the programming of both agents, A1 agent not only will yield a superior global performance but that will reach goal R before A0 in a greater number of occasions. This is clearly in consonance with the results of Gollwitzer and collaborators about the superiority to plan in humans the actions by means of implementation intentions as opposed to the mere attempt to execute a goal intention.

Gollwitzer and Sheeran (2004) have made a meta-analytical study of the effects exerted by the formulation of implementation intentions in the behavior of achievement of goals on the part of the agents. We set out to transfer the fundamental parameters with humans to A1 agent and to compare results with A0 agent more oriented to the execution of the goal intention to reach R. According to authors (2004, p. 26), the general impact of implementation intentions on the achievement of goals is of d=0.65, based on k=94 tests that implied 8461 participants. An important effect (op. cit., p. 29) was obtained for implementation intentions when the achievement of goals was blocked by adverse contextual influences (d=0.93). The accessibility to the situational cues was of d=0.95. To A1 we have assigned a 65 percent of percentage in the achievement of the goal. We have located a difference of 30 points in the achievement of R and according to a difference of percentage in the achievement of R on the part of A0 of 16 points, A0 was assigned a degree of achievement of 81 percent. As for the accessibility of the situational cues L, this one is very high in A1 agent (95) and considering that A1 can add 30 points more than A0, taking advantage of the situations, we have assigned to A0 a percentage of accessibility of the 76 percent. Considering that the degree of avoidance of obstacles S on the part of A1 is very high (93), to A0, we have assigned a difference to it of 19 points, that is to say, of the 74 percent. However, to fall in anyone of places S counts equal reason why it affects to the penalty for both agents.

We give account of the results, once made 5000 trials, with an average of about 48 movements by trial. We have considered the total number of plays, points (average), total resumptions (average), total victories or the number of times that the agent reaches R in the first place, number of situational cues L, number of obstacles S and number of carried out movements. The system of assigned points was:

A0: start: +50; L0-L5: +20; S0-S5: -5; R: +150; D0 (dissuasive agent intercepting the agents A0 and A1): -150; penalty by each movement: -1.

A1: start: +50; L0-L5: +25: S0-S5: -5; R: +120; D0 (dissuasive agent intercepting the agents A0 and A1): -150; penalty by each movement: -1.

The diversity of tasks that the agents have to execute in the board ends up interacting of a dynamic and significant way. This still is appraised with greater forcefulness in the one perhaps that it is the most decisive and surprising result of this exercise of simulation: the one that the most planning agent A1 achieves goal R in a greater percentage of times than A0, when A0 has been programmed to perceive and to accede to R with greater facility. We believe that our simulation has fulfilled the basic objective of supporting, in the area of Artificial Intelligence, the experimental conclusions with humans, of Gollwitzer and other authors about the superiority of the use of implementation intentions in the goal achievement, against the emphasis located in the execution of the goal intentions. As an obvious result, this task, given its limited nature, has not collected all the possibilities. Thus, the issue of the beginning of goal purpose has not been approached. Neither has the issue of the fact that the agents abandon the purpose of reaching R or that they seek alternative goals. On the other hand, not even the effect on the learning of the task as consequence of successive frustrations has been outlined. It would be interesting, to introduce agents not only based on learning rules but also adaptive agents.