PoCoMo: playful social interactions between multiple projected characters

PoCoMo (see http://fluid.media.mit.edu/people/roy/media/PoCoMo-cam-ready-optimized.pdf) consists of two mobile projector-camera systems with computer vision algorithms to support the behaviours of characters projected in the environment. The characters are guided by hand movements and can respond to other characters, simulating a reality of life-like agents. Users hold micro projector-camera devices to project animated characters on the wall and the characters recognize and interact with one another. Extracting visual features from the environment, an algorithm enables operations in a limited resource environment. The system creates games including social scenarios of relating and exchange between to co-located users. The characters are programmed to have component parts with separate articulation and with different sequences based on the proximity of the other characters. The projected characters can respond to the presence and the orientation of one another and acknowledge each other. Also they can trigger gestures of friendship such as shaking hands. Characters can leave presents to one another. Each character has an identity that is represented by the color of its markers. A detection algorithm scans the image by applying a threshold extracting the contour of the figures. The detection algorithm has been implemented in C++ and compiled to a native library. The UI of the application has been implemented in JAVA using ANDROID API. In a future work, the authors (Shilkrot, Hunter and Maes from MIT Media Lab) will integrate markers with the projected content and migrate the application to devices with wider fields of views.

Gaze following and mirror neurons

Gaze following is a basic component of the human social interaction and it is a type of attention sharing behaviors. It is also present in a number of other species (for instance, apes) and seems necessary for designing social robots. It can be defined as the ability to look where somebody else is looking. Triesch, Jasso and Deák (2007) have formulated a computational model of gaze following by means of ideas related to the behavior of mirror neurons. The authors emphasize the role of learning processes by means of the interaction with the social environment. In fact, gaze following can be linked to imitation. The link between gaze following and imitation also implies the similarity between the neural basis of gaze following and the neural basis of other imitative behaviors. Triesch and collaborators develop a model that share properties with mirror neurons, that is, neurons implicated in imitation and originally founded in macaque area F5 by Rizzolatti and his team in Parma University. The model predicts the existence of a new class of mirror neurons for looking behaviors that has not been observed experimentally. In this model, an infant and a caregiver interact with a number of visually salient objects. During the process, the infant learns to predict the locations of salient objects based on the looking behavior of the caregiver. There are periods when the caregiver is present and periods when the infant is alone with the objects. When the caregiver is present, the infant and caregiver are in fixed locations facing each other with a separation between them. At any time a random number of objects will be present. Habituation decreases the perceived saliency of an object.

The infant model learns through a reinforcement learning scheme. The learning process tries to optimize the infant´s policy, that is, the way the actor maps sensory states onto different gaze shifts in order to maximize the long-term reward obtained by the infant. Reward is obtained as the saliency of the position to which attention is directed after a gaze shift has been made. At each time step, the caregiver looks at the most salient object, where saliency is mediated by the same habituation mechanism as in the infant´s visual system. The model neurons in the pre-motor layer share many characteristics with classical mirror neurons. A unit in this layer will be active during the execution of a gaze shift to a certain location in space. This is because the probability of performing such a gaze shift is related to the activation of the unit. The units in the layer will be active when the infant observes the caregiver looking in the corresponding direction. Clearly, the neurons in the pre-motor layer can be viewed as mirror neurons because the combination of being active during execution and observation of a motor act is the defining characteristic of mirror neurons.

Following to the authors, this model can be considered a simple associative learning account of a response facilitation but also has implications for the question of whether mirror neurons are innate or whether they acquire their properties through a learning process. For the mirror neurons concerned with grasping, they find plausible that there are situations where observing an agent grasp an object may predict a reward if the same action is attempted. Such situations may be sufficient for the emergence of mirror neurons for grasping. The reviewed model predicts a very close connection between mirror neurons and imitative behaviors.