TY  - CHAP
U1  - Teil eines Buches
A1  - Richter, Mats L.
A1  - Schöning, Julius
A1  - Richter (geb. Wiedenroth), Anna
A1  - Krumnack, Ulf
T1  - Receptive Field Analysis for Optimizing Convolutional Neural Network Architectures Without Training
T2  - Deep Learning Applications, Volume 4
N2  - The number of layers in convolutional neural networks (CNN) is often overshot, when a convolutional neural network architecture is designed for an image-based task. These CNN-architectures are therefore unnecessarily costly to train and deploy. The increase in network-complexity also results in diminishing returns in terms of the predictive quality. The receptive field of a convolutional layer is strictly limiting the features it can process. We can consistently predict unproductive layers that will not contribute qualitatively to the test performance in a given CNN architecture, by analyzing the receptive field expansion over the network. Since the receptive field is a property of the architecture itself, this analysis does not require training the model. We refer to this analysis technique as Receptive Field Analysis (RFA). In this work, we demonstrate that RFA can be used to guide the optimization of CNN architectures by predicting the presence of unproductive layers. We show that RFA allows the deduction of design decisions and simple design strategies that reliably improve the parameter efficiency of the model on the given task. We further demonstrate that these RFA-guided strategies can reliably improve the predictive performance, computational efficiency or strike a balance between the two. Finally, we show that RFA can also be used to define an interval of feasible input resolutions for any modern architecture, in which the model will operate with high efficiency, while being able to extract any pattern from the image. This allows practitioners to pick efficient input resolutions when adapting models for novel tasks.
KW  - Receptive field size
KW  - Optimization
KW  - Neural architecture design
KW  - Trainable parameter
KW  - Computational efficiency
KW  - Input resolution
Y1  - 2023
SN  - 978-981-19-6152-6
SB  - 978-981-19-6152-6
U6  - https://doi.org/10.1007/978-981-19-6153-3_10
DO  - https://doi.org/10.1007/978-981-19-6153-3_10
N1  - Zugriff im Hochschulnetz
SP  - 235
EP  - 261
PB  - Springer Nature
CY  - Singapore
ER  -