Just announced Microsoft Translator enhancements speed up translation times

Maourice Gonzalez

Microsoft Translator Enhancements

Looking for more info on AI, Bing Chat, Chat GPT, or Microsoft's Copilots? Check out our AI / Copilot page for the latest builds from all the channels, information on the program, links, and more!

Microsoft just announced the adoption of Z-Code Mixture of Experts to its Azure Translation Cognitive Service. This is the first publicly available service on Azure to leverage this approach.

What is Mixture of Experts?

Z-code models employ a new design known as Mixture of Experts (MoE), in which different elements of the models can learn to do different tasks. The models learn to translate between numerous languages simultaneously. The Z-code MoE model employs additional parameters while dynamically determining which parameters to apply to a particular input. During training, the model might specialize a subset of the parameters (experts). The model uses the appropriate experts for the task at runtime, which is more computationally efficient than using all of the model’s parameters.

It is difficult to train huge models with billions of parameters. The Translator team worked with Microsoft DeepSpeed to create a high-performance system that assisted in the training of enormous scale Z-code MoE models, allowing them to efficiently scale and deploy Z-code models for translation.

Zcode_MoE_v2-final_GIF
Z-code MoE model translating from English to French. The model dynamically selects subsets of its parameters to be utilized for each input.

 

To make this possible Microsoft partnered with NVIDIA to optimize engines that can be used at runtime to deploy the new models which yielded a substantial 27x improvement over the previous implementation.

To read more about this Microsoft Translator and Z-Code Mixture of Experts, head over to the official announcement post from Microsoft.