--- title: Granularity keywords: fastai sidebar: home_sidebar summary: "What block of parameter to remove in a neural network ?" description: "What block of parameter to remove in a neural network ?" nb_path: "nbs/0a_granularity.ipynb" ---
A Conv2d
layer possess a 4d-tensor as weights. This means that there exist many ways of removing blocks from it.
In the case of convolution filters, removing 0-D elements is equivalent to removing individual weights.
weight
granularityget_pruned_conv(0)
1-D blocks of elements is equivalent to removing vectors from the convolution filters. There are several ways to chose the vectors, that will be represented below.
shared_weight
: this granularity is very particular as it removes individual weights from a filter, but with a pattern that is shared across all filters.get_pruned_conv(1)
channel
: remove vector of weights along the channel axis.get_pruned_conv(2)
column
: remove vector of weights along the height axis.get_pruned_conv(3)
row
: remove vector of weights along the width axis.get_pruned_conv(4)
shared_channel
: remove vector of weight along the channel axis, but with a pattern that is shared across all filters.get_pruned_conv((1,2))
shared_column
: remove vector of weight along the height axis, but with a pattern that is shared across all filters.get_pruned_conv((1,3))
shared_row
: remove vector of weight along the width axis, but with a pattern that is shared across all filters.get_pruned_conv((1,4))
vertical_slice
: remove vertical slices of weight along the height axis.get_pruned_conv((2,3))
horizontal_slice
: remove vertical slices of weight along the width axis.get_pruned_conv((2,4))
kernel
: remove kernels of from the convolution filters.get_pruned_conv((3,4))
shared_vertical_slice
: remove vertical slices of weight along the height axis, with a pattern that is shared across all filters.get_pruned_conv((1,2,3))
shared_horizontal_slice
: remove horizontal slices of weight along the width axis, with a pattern that is shared across all filters.get_pruned_conv((1,2,4))
shared_kernel
: remove kernels of weight from the convolution filters, with a pattern that is shared across all filters.get_pruned_conv((1,3,4))
filter
: remove entire filters.get_pruned_conv((2,3,4))
As for the convolution filters, weights from a Linear layer can be removed independently.
weight
: remove individual weights.get_pruned_linear(0)
column
: remove column of weight, which corresponds to removing input neurons.get_pruned_linear(1)
row
: remove rows of weight, which corresponds to removing output neurons.get_pruned_linear(2)
{% include note.html content='This is an experimental part of the library' %}