![]() ![]() ![]() Learning from these initial results and after a lot of iteration we discovered a recipe that allows our simple student to have almost the same quality as the complex teacher (sometimes there is a free lunch after all?). Now we were free to build large, complex teacher models to maximize quality, without worrying about real-time constraints (too much). 2017) models and their modifications play well with teacher-student training and are astoundingly efficient during inference on the CPU. 2018a). Some particularly exciting results from this effort were that Transformer ( Vaswani et al. This is a simpler task than learning from raw data, and allows a shallower, simpler student to very closely follow the complex teacher. As one might expect, our initial attempts still suffered quality drops from teacher to student (no free lunch!), but we nevertheless took first place in the WNMT 2018 Shared Task on Efficient Decoding ( Junczys-Dowmunt et al. Our first step was to switch to a “teacher-student” framework, where we train a lightweight real-time student to mimic a heavyweight teacher network ( Ba and Caruana 2014). This is accomplished by training the student not on the parallel data that MT systems are usually trained on, but on translations produced by the teacher ( Kim and Rush 2016). ![]() Over the past year, we have been looking for ways to bring much of the quality of our human-parity system into the Microsoft Translator API, while continuing to offer low-cost real-time translation. Here are some of the steps on that journey. These new models are available today in Chinese, German, French, Hindi, Italian, Spanish, Japanese, Korean, and Russian, from and to English. These models incorporate most of the goodness of our research system and are now available by default when you use the Microsoft Translator API. Today we are excited to announce the availability in production of our latest generation of neural Machine Translation models. This was an exciting breakthrough in Machine Translation research, but the system we built for this project was a complex, heavyweight research system, incorporating multiple cutting-edge techniques. While we released the output of this system on several test sets, the system itself was not suitable for deployment in a real-time machine translation cloud API. 2018) a breakthrough result where we showed for the first time a Machine Translation system that could perform as well as human translators (in a specific scenario – Chinese-English news translation). In March 2018 we announced ( Hassan et al. ![]()
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |