Learning To Recover From Multi-modality Errors For Non-autoregressive Neural Machine Translation
Qiu Ran, Yankai Lin, Peng Li, Jie Zhou . Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 2020 – 40 citations
[Paper]
Non-autoregressive neural machine translation (NAT) predicts the entire target sequence simultaneously and significantly accelerates inference process. However, NAT discards the dependency information in a sentence, and thus inevitably suffers from the multi-modality problem: the target tokens may be provided by different possible translations, often causing token repetitions or missing. To alleviate this problem, we propose a novel semi-autoregressive model RecoverSAT in this work, which generates a translation as a sequence of segments. The segments are generated simultaneously while each segment is predicted token-by-token. By dynamically determining segment length and deleting repetitive segments, RecoverSAT is capable of recovering from repetitive and missing token errors. Experimental results on three widely-used benchmark datasets show that our proposed model achieves more than 4(\times) speedup while maintaining comparable performance compared with the corresponding autoregressive model.