International Journal of Hybrid Intelligent Systems

International Journal of Hybrid Intelligent Systems

Volume 1, No. 2 (2004)

Backward Elimination Methods for Associative Memory Network Pruning

Xia Hong, Chris Harris, Martin Brown and Sheng Chen

Abstract. Three hybrid data based model construction/pruning formula are introduced by using backward elimination as automatic postprocessing approaches to improved model sparsity. Each of these approaches is based on a composite cost function between the model fit and one of three terms of A-/D-optimality / (parameter 1-norm in basis pursuit) that determines a pruning process. The A-/D-optimality based pruning formula contain some orthogonalisation between the pruned model and the deleted regressor. The basis pursuit cost function is derived as a simple formula without need for an orthogonalisation process. These different approaches to parsimonious data based modelling are applied to the same numerical examples in parallel to demonstrate their computational effectiveness.

Keywords: nonlinear modelling, backward elimination, forward regression, model sparsity, generalization.