We present a measure of representation in neural networks that we call ‘R’, which is based on information theory. We show how R relates to an analysis of distributed representation, viz. a principal components analysis of activation space. Finally, we argue that R is well suited to measure representation in neural networks.
Copyright 2009 by the Australasian Society for Cognitive Science. Publisher version archived with the permission of the Editor, ASCS09 : Proceedings of the 9th Conference of the Australasian Society for Cognitive Science. This copy is available for individual, non-commercial use. Permission to reprint/republish this version for other uses must be obtained from the publisher.