AMBER (AlignedMultilingual Bidirectional EncodeR) is trained on additional paralleldata using two explicit alignment objectives that align the multilingualrepresentations at different granularities . AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model which has 4.6x theparameters of AMBER . Experimental results show thatAMBER . obtains . gains of . up to . 1,000% F1 scores on sequences tagging, .sentence retrieval and sentence classification and . sentence classification over XLMR large model with 4x the parameters of AMber .

Author(s) : Junjie Hu, Melvin Johnson, Orhan Firat, Aditya Siddhant, Graham Neubig

Links : PDF - Abstract

Code :


Keywords : amber - sentence - model - average - gains -

Leave a Reply

Your email address will not be published. Required fields are marked *