Sandwich Batch Normalization

Sandwich Batch Normalization ($\textbf{SaBN}$) is a frustratingly easy improvement of Batch normalization . SaBN factorizes the BN affine layer into one shared layer, cascaded by several parallel parallel layers . Its variants include further decomposing the normalization layer into multiple parallel ones, and extending similar ideas to instance normalization. We demonstrate the prevailing effectiveness of SaBN (as well as its variants) as a drop-in replacement in four tasks: neural architecture search (NAS), image generation, adversarial training, and style transfer. We also provide visualizations and analysis to help understand why SaBN works. All our codes and pre-trained models will be released upon acceptance. All of our codes will be publicly available upon acceptance, including those that have been accepted by The Open Beta Software Institute. Back to the page you came from.

Links: PDF - Abstract

Code :


Keywords : normalization - sabn - batch - parallel - layer -

Leave a Reply

Your email address will not be published. Required fields are marked *