Illinois Genetic Algorithms Laboratory

Evolutionary Computation in Conceptual Clustering and Tagging

admin — Mon, 19 May 2008 15:38:15 +0000

Abstract: The Web 2.0 technologies provide users with collaborative work-spaces over the Internet. For example, Wikipedia is an open source encyclopedia that anyone can edit articles. YouTube provides spaces where users can share videos and annotations about them. Users can put images on Flickers and collaborate each other by categorizing with tagging. These contents are created by users’ voluntary activities, which is one of the features for the Web 2.0 technology. Some services based on the Web 2.0 have well organized text-based contents on them. For example, Wikipedia has well structured contents, which is due to voluntary activities of the users trying to edit each contents so as to be sophisticated. On the other hands, other services, such as YouTube and Flickers’, only have short sentences or small number of words as annotations. Additionally these annotations are usually different according to each user because participants are not in situation of collaborations. As a result, annotations do not have meaning for videos and pictures. A system that converts annotations into meaningful texts is useful because building texts requires resources while a number of annotations are available.The purpose of this thesis is development of the text builder which is based on the Web 2.0 technology with genetic algorithms and natural language processing. A human interactions system is developed in this thesis for automatically building meaningful tags from annotations. The system consists of mainly two parts: a conceptual clustering component based on natural language processing and a sentence creating component based on genetic algorithms. The conceptual clustering component decomposes annotations into phrases with main ideas. Then, the sentence creating component builds several tags from the phrases. Thirdly created tags are evaluated by users to find better explanations for videos and pictures. Participants are supposed to collaborate through evaluations to create more organized and meaningful tags.The developed system succeed in creating tags from annotations without structures through user-machine interactions. This system is applicable to other systems which has only annotations as participants’ comments. Because tags created by this system have meanings but short length a system building longer text as sentences is left as future works.

An Analysis of Matching in Learning Classifier Systems

admin — Thu, 15 May 2008 19:40:04 +0000

Abstract: We investigate rule matching in learning classifier systems for problems involving binary and real inputs. We consider three rule encodings: the widely used character-based encoding, a specificity-based encoding, and a binary encoding used in Alecsys. We compare the performance of the three algorithms both on matching alone and on typical test problems. The results on matching alone show that the population generality influences the performance of the matching algorithms based on string representations in different ways. Character-based encoding becomes slower and slower and generality increases, specificity-based encoding becomes faster and faster as generality increases. The results on typical test problems show that the specificity-based representation can halve the time require for matching but also that binary encoding is about ten times faster on the most difficult problems. Moreover, we extend specificity-based encoding to real-inputs and propose an algorithm that can halve the time require for matching real inputs using an interval-based representation.

Investigating Restricted Tournament Replacement in ECGA for Non-Stationary Environments

admin — Tue, 06 May 2008 06:07:47 +0000

Abstract: This paper investigates the incorporation of restricted tournament replacement (RTR) in the extended compact genetic algorithm (ECGA) for solving problems with non-stationary optima. RTR is a simple yet efficient niching method used to maintain diversity in a population of individuals. While the original version of RTR uses Hamming distance to quantify similarity between individuals, we propose an alternative substructural distance to enforce the niches. The ECGA that restarts the search after a change of environment is compared with the approach of maintaining diversity, using both versions of RTR. Results on several dynamic decomposable test problems demonstrate the usefulness of maintaining diversity throughout the run over the approach of restarting the search from scratch at each change. Furthermore, by maintaining diversity no additional mechanisms are required to detect the change of environment, which is typically a problem-dependent and non-trivial task.

Self-Adaptive Mutation in XCSF

admin — Wed, 30 Apr 2008 02:00:44 +0000

Abstract: Recent advances in XCS technology have shown that self-adaptive mutation can be highly useful to speed-up the evolutionary progress in XCS. Moreover, recent publications have shown that XCS can also be successfully applied to challenging real-valued domains including datamining, function approximation, and clustering. In this paper, we combine these two advances and investigate self-adaptive mutation in the XCS system for function approximation with hyperellipsoidal condition structures, referred to as XCSF in this paper. It has been shown that XCSF solves function approximation problems with an accuracy, noise robustness, and generalization capability comparable to other statistical machine learning techniques and that XCSF outperforms simple clustering techniques to which linear approximations are added. This paper shows that the right type of self-adaptive mutation can further improve XCSF’s performance solving problems more parameter independent and more reliably. We analyze various types of self-adaptive mutation and show that XCSF with self-adaptive mutation ranges, differentiated for the separate classifier condition values, yields most robust performance results. Future work may further investigate the properties of the self-adaptive values and may integrate advanced self-adaptation techniques.

Real-Coded Extended Compact Genetic Algorithm based on Mixtures of Models

admin — Wed, 23 Apr 2008 04:23:12 +0000

Abstract: This paper presents a real-coded estimation distribution algorithm (EDA) inspired to the extended compact genetic algorithm (ECGA) and the real-coded Bayesian Optimization Algorithm (rBOA). Like ECGA, the proposed algorithm partitions the problem variables into a set of clusters that are manipulated as independent variables and estimates the population distribution using marginal product models (MPMs); like rBOA, it employs finite mixtures of models and it does not use any sort of discretization. Accordingly, the proposed real-coded EDA can be either viewed as the extension of the ECGA to real-valued domains by means of finite mixture models or as a simplification of the real-coded BOA to the marginal product models (MPMs). The results reported here show that the number of evaluations required by the proposed algorithm scales sub-quadratically with the problem size in additively separable problems.

Sequential Problems that Challenge Generalization in Classifier Systems

admin — Wed, 23 Apr 2008 04:11:25 +0000

We present an approach to build sequential decision making problems which can challenge the generalization capabilities of classifier systems. The approach can be applied to any sequential problem defined over a binary domain and it generates a new problem with bounded sequential difficulty and bounded generalization difficulty. As an example, the approach is used here to generate two problems with a simple sequential structure, huge number of states (more than a million), and many generalizations. These problems are used to compare a classifier system with effective generalization (XCS) and a learner without generalization (Q-learning). The experimental results confirm what was previously found mainly using single-step problems, also in sequential problems with huge state spaces, XCS can generalize effectively by detecting those context-dependent structures that are necessary for optimal sequential behavior.

Enhancing the Efficiency of the ECGA

admin — Thu, 03 Apr 2008 15:02:00 +0000

Abstract: Evolutionary Algorithms are largely used search and optimization procedures that, when properly designed, can solve intractable problems in tractable polynomial time. Efficiency enhancements are used to turn them from tractable to practical.

In this paper we show preliminary results of two efficiency enhancements proposed for the Extended Compact Genetic Algorithm. First, a model building enhancement was used to reduce the complexity of the process from O(n^3) to O(n^2), speeding up the algorithm by 1000 times on a 4096 bits problem. Then, local-search hybridization was used to reduce the population size by at least 32 times, reducing the memory and running time required by the algorithm. These results draw the first steps toward a competent and efficient Genetic Algorithm.

Linkage Learning, Rule Representation, and the χ-ary Extended Compact Classifier System

admin — Tue, 18 Mar 2008 20:04:54 +0000

Abstract: This paper reviews a competent Pittsburgh LCS that automatically mines important substructures of the underlying problems and takes problems that were intractable with first-generation Pittsburgh LCS and renders them tractable. Specifically, we propose a χ-ary extended compact classifier system which uses (1) a competent genetic algorithm (GA) in the form of $\chi$-ary extended compact genetic algorithm, and (2) a niching method in the form restricted tournament replacement, to evolve a set of maximally accurate and maximally general rules. Besides showing that linkage exist on the multiplexer problem, and that χeCCS scales exponentially with the number of address bits (building block size) and quadratically with the problem size, this paper also explores non-traditional rule encodings. Gene expression encodings, such as the Karva language, can also be used to build χeCCS probabilistic models. However, results show that the traditional ternary encoding { 0,1,#} presents a better scalability than the gene expression inspired ones.

Speeding Online Synthesis via Enforced Selecto-Recombination

admin — Mon, 17 Mar 2008 17:27:26 +0000

Abstract: Brainstorming has been greatly used as a method to generate a large number of ideas by variety of each participant’s knowledge. However, brainstorming does not always work well because of spatial, communication limitations. Moreover, brainstorming techniques present limited scalability. Meanwhile, genetics algorithms have been mostly regarded as an engineering or technological tool. However, the innovation intuition suggests that genetic algorithms may be also regarded as models of human innovation and creativity. This paper focuses on online creativity sessions. Modeling those creative efforts using selecto-recombinative mechanism can provide three times more novel ideas, increase the posting frequency by a 2.6 factor, and help overcome superficiality on online communications by favoring synthetic thinking.

Graph-Theoretic Measure for Active iGAs: Interaction Sizing and Parallel Evaluation Ensemble

admin — Mon, 10 Mar 2008 20:29:58 +0000

Abstract: Since their inception, active interactive genetic algorithms have successfully combat user evaluation fatigue induced by repetitive evaluation. Their success originates on building models of the user preferences based on partial-order graphs to create a numeric synthetic fitness. Active interactive genetic algorithms can easily reduce up to seven times the number of evaluations required from the user by optimizing such a synthetic fitness. However, despite basic understanding of the underlying mechanisms, active interactive genetic algorithms still lack of principled understanding of what properties make a partial ordering graph a successful model of user preferences. Also, there has been little research conducted about how to integrate together the contribution of different users to successfully capitalize on parallelized evaluation schemes. This paper addresses both issues describing (1) what properties make a partial-order graph a success, and (2) how partial-order graphs obtained from different users can be merged meaningfully.