RT-58
Abstract

Artificial neural networks have been applied to a variety of classification and learning tasks. The success of error correction training algorithms such as backpropagation has meant that supervised learning, where the correct outcome is known, has been the most used. In addition, although the structure of an artificial neural network is a significant contributing factor to its performance, the structure is generally heuristically chosen. The use of evolutionary algorithms as search techniques has allowed different properties of artificial neural networks to be evolved. Several successful neural/evolutionary approaches such as the EPNet algorithm are hybrid algorithms combining gradient descent methods such as backpropagation learning for connection weight training with evolutionary structure search. Generally the fitness measure used in these approaches is a mean or sum squared network output error over input training patterns. This type of fitness measure is applicable to supervised learning, but not necessarily to reinforcement learning tasks where the correct output is not generally known. Evolutionary search algorithms allow greater flexibility in the choice of a suitable fitness measure. As a result, neural/evolutionary algorithms that are not reliant on gradient descent techniques and not reliant on the standard fitness measure may have wider applicability outside the supervised learning domain.

The goal of this work is the implementation of an evolutionary algorithm that simultaneously searches for optimum artificial neural network structures and weights without the use a gradient-descent process. Neural network structural modifications are alternated with connection weight search/training attempting minimum disruption between parent and offspring neural network individuals whilst maintaining an effective search.

The implemented algorithm is heavily based on the successful EPNet algorithm, applying mutation operators in a step-wise process, but does not use a backpropagation weight training algorithm. Evolving Neural Network Architecture and Weights Using An Evolutionary Algorithm Two different fitness functions are used - a standard sum squared error calculation and an accuracy calculation based on correct pattern classifications and network complexity. The aim of this is to enable a comparison of the effectiveness of the two fitness functions on the same problems. A secondary aim of this work is to determine if the implemented algorithm will continue to find smaller, less complex neural networks that generalise better after a first solution network has been found. The algorithm is tested on Even Parity 4 and 5 and the IRIS data classification problems. The results show that solution networks with low complexity can be found for Parity 4 and 5 problems using the accuracy fitness. The mean squared output accuracy performance of the solution networks was worse in comparison to the results of EPNet. Where solution networks were found early in a run, the final solutions were always less complex than the first solutions. This indicates that the use of a network complexity penalty incorporated into the fitness function was successful. The results on the IRIS data classification problem were good, however the algorithm needs to be applied to more difficult classification problems to test its scalability. Experiments on the parity problems with the standard sum squared error function were disappointing with only one solution being found for parity 4. The conclusion of this work is that the accuracy fitness was more effective than the sum squared error fitness calculation in finding solution networks on the tasks attempted. Further research is required to determine if the ineffectiveness of the error function is a result of the disruptive effect of an overly weighted network complexity penalty. The results indicate that neural networks that solve problems can be found without using gradient descent algorithms and the standard sum squared error fitness measure.