Origin of epistemic structures and computational agents

Adding cognitive structures to the world is a basic adaptive strategy. Chandrasekharan and Stewart (2007) developed an interesting perspective modeling swarms of foraging robots for simulating epistemic structures in agents. Epistemic structures could be structures generated for oneself (for instance, bookmarks), structures generated for oneself and others (pheromones, etc.) and structures generated exclusively for others (warning smells...) (see Chandrasekharan and Stewart, 2007, 331). A key feature of such structures is their task-specifity. According to Chandrasekharan and Stewart (art. cit., 332), organisms sometimes generate random structures in the environment and organisms have a bias to reduce physical or cognitive effort. The authors design a computational simulation consisting of an environment of a 30x30 toroidal gridworld, with one 3x3 square patch representing the agent´s home, and another representing the target. This target can be thought of as a food source. An agen can perform several possible actions: moving randomly; to distinguish between their home and their target (consider models of ant foraging and postulate a home pheromone and a food pheromone) and to modify the environment. The agents have four sensors, two external and two internal and are programmed by a genetic algorithm to evolve foraging behavior in the agents. The fitness function of the genetic algorithm is the inverse of the time measure, which is interpreted as an expression of tiredness. In the simulation, 10 agents foraged at the same time. Initially, the agents behaved randomly. Most agents did not find the target. On average, each agent was completing 0.07 foraging trips every 100 time steps. After a few hundred generations, the agents were completing an average of 1.9 trips in that same period. The result confirmed that the agents were able to systematically make use of their ability to sense and generate structures in the world, on an evolutionary time scale. But is possible for the agents adding structures to the world within their lifetimes? The Q-learning (Watkins, 1989) is the simplest method to perform this task. Using the Q-learning algorithm, 10 agents were ran for 1,000 time steps. To indicate tiredness, it was gave them a reinforcement value of -1 while foraging. When they returned home after finding the target, they were given a reinforcement of 0. By the end of the simulation, agents required only around 150 time steps to make a complete trip (a foraging rate of 0.66 trips in 100 time steps, that is, twice as quick as agents without the structure-forming ability). Even they spent 58% of their time generating structures: epistemic structures generation allowed the agents to complete their foraging task down from 300 time steps. These simulations facilitate an integration of the symbolic and situated views of cognition supporting the extended mind thesis.