Channel-wise Convolution

Channelwise (Depthwise) Convolution layer for a sparse tensor.

\[\mathbf{x}_\mathbf{u} = \sum_{\mathbf{i} \in \mathcal{N}^D(\mathbf{u}, K) \cap \mathcal{C}^\text{in}} W_\mathbf{i} \odot \mathbf{x}_{\mathbf{i} + \mathbf{u}} \;\text{for} \; \mathbf{u} \in \mathcal{C}^\text{out}\]

where \(K\) is the kernel size and \(\mathcal{N}^D(\mathbf{u}, K) \cap \mathcal{C}^\text{in}\) is the set of offsets that are at most \(\left \lceil{\frac{1}{2}(K - 1)} \right \rceil\) away from \(\mathbf{u}\) defined in \(\mathcal{S}^\text{in}\). \(\odot\) indicates the elementwise product.

For even \(K\), the kernel offset \(\mathcal{N}^D\) implementation is different from the above definition. The offsets range from \(\mathbf{i} \in [0, K)^D, \; \mathbf{i} \in \mathbb{Z}_+^D\).

Written on January 5, 2020