Illinois Genetic Algorithms Laboratory

Butz, M. V., Goldberg, D. E. (2002)
TR No.: 2002019 | Download PDF | Download PS

Abstract:
This paper introduces generalized state values to the anticipatory learning classifier system ACS2. Previous studies with ACS2 showed that the system reliably evolves a generalized predictive model in typical Markov decision process (MDP). The predictive model approximates the state transition function of the MDP in a compact, generalized form. However, it was also shown that the evolving predictive model might be over-general for an accurate representation of reinforcement values. Thus, a function approximation module is added that approximates state values. In combination, actual action choice depends on state values predicted by the means of the predictive model yielding anticipatory behavior. It is shown that the function approximation module accurately generalizes the state value function in the investigated MDP. We also suggest the implementation of task dependent anticipatory attentional mechanisms exploiting the representation of the generalized state-value function. Moreover, improvement of the approach by the means of further anticipatory interaction between predictive model learner and state value learner is suggested.