Parameterization of Hypercomplex Multiplications

The Hamilton product (4D hypercomplex multiplication) enables learning effective representations while saving up to 75% parameters… However, one key caveat is that hypercomplex space only exists at very few predefined dimensions . This restricts the flexibility of models that leverage hypercomplex multiplications . To this end, we propose parameterizing hypercomplex . multiplications, allowing models to learn multiplication rules from data regardless of whether such rules are predefined . As a result, our method not only subsumes the Hamilton product, but also learns to operate on any arbitrary nD hyper complex space, providing more architectural flexibility . Experiments of applications to LSTM and Transformer on natural language inference, machine translation, text style transfer, and subject verb agreement demonstrate .

Links: PDF - Abstract

Code :


Keywords : hypercomplex - multiplications - predefined - hamilton - space -

Leave a Reply

Your email address will not be published. Required fields are marked *