EFFICIENT ARCHITECTURE OF ARTIFICIAL NEURAL NETWORKS

Melissa Utzinger

This Summer I experimented with Artificial Neural Networks (ANNs) that learn computer programs modeled after the human brainís neurological system. If ANNs are implemented successfully, then there could be dramatic technological advancements in machine learning to reduce credit card fraud, to diagnose diseases, to forecast future sales or to predict stock market behavior. The network structure is a set of inputs, with weighted connections to a layer of hidden processing nodes. These are connected by weights to an output node.

There are three basic types of learning that are used to train ANNs: supervised learning, unsupervised learning and reinforcement learning. I worked with supervised learning, specifically learning using back propagation. Before a neural network can work it must go through the process of learning. In supervised learning this is done through example. Back propagation neural networks consist of a forward step and a backward step. Inputs are presented to the neural network, which then permeate through the network and result in an output (feed-forward step). This output is then compared with the desired output in order to calculate an error. The error filters backward through the neural network as the ANN weights are adjusted accordingly (feed-backward step).

I worked on creating a neural network that learns the XOR logic function. I was able to experiment with the learning rate of the ANN, the number of iterations used to train the ANN, and the number of hidden nodes in the ANN structure. I found 3000 iterations with a 0.5 learning rate served as a good comparison for experimenting with the number of hidden nodes. The two diagrams below represent the squared error of two different neural networks. A neural network with 13 hidden nodes learns faster than the neural network with 2 nodes, the minimum needed for XOR. There was little difference using greater than 13 hidden nodes.

ANNs are making progress in the technology field, both in hardware and software. Their commercial applications are already underway and with future research and discoveries more efficiently designed neural networks can advance and improve our society. (Supported by the Schultz Fellowship).

Figure 1: Thirteen hidden nodes Figure 2: Two Hidden nodes

Advisor: Professor Judy Franklin