Researcher’s review of the 16th MT Marathon: machine translation trends and new tools to keep an eye on
Team Tilde April 9, 2024
I am Toms Bergmanis, a machine translation (MT) researcher and practitioner at Tilde. My day-to-day job involves building custom MT systems for large international organisations, as well as developing novel solutions that enable users to control and customise MT systems at runtime.
I recently attended this year’s Machine Translation Marathon to broaden my knowledge and mingle with other researchers and practitioners from academia and industry. I really enjoyed my time there and found it very beneficial, so I decided to share my experience of the event and some of the work presented.
MT Marathon is a week-long event, typically hosted by Charles University in Prague or my alma mater, the University of Edinburgh. This year, however, it was organised by Lisa Yankovskaya and Mark Fishel in the charming city of Tartu in Estonia.
The 16th MT Marathon continued its tradition of offering a comprehensive programme catering to all levels of expertise. It featured foundational MT lectures and hands-on labs for beginners, followed by engaging keynote talks by renowned researchers and practitioners from academia and industry. Videos and lecture slides are available online.
This year’s event had lots of talks on MT quality evaluation. Maja Popović from Dublin City University, Nuno Guerreiro from Unbabel, and Samuel Läubli from Textshuttle gave presentations on manual and automatic evaluation, quality estimation and challenges comparing MT and human translation performance. This reflects the overall trends in the industry conferences, where MT evaluation has seen increased interest in recent years. As no Natural Language Processing (NLP) event in 2023 wouldn’t be complete without a talk on large generative language models, Alexandra Birch from the University of Edinburgh answered the question of whether they will replace traditional MT.
MT Marathon also offers space for collaboration on self-proposed research and software development projects. My favourites, however, were talks presenting new, recently developed tools enabling future research and MT system production.
One of them, the MAMMOTH toolkit, was presented by Raúl Vázquez and Timothee Mickus from Jörg Tiedemann’s group at the University of Helsinki. MAMMOTH stems from the Helsinki NLP group’s research on MT models that translate between more than just two languages. The acronym stands for MAssively Multilingual Modular Open Translation @ Helsinki and aims to solve challenges arising from the large models being trained on vast amounts of data. MAMMOTH’s authors have done a colossal work implementing all imaginable flavours of parameter sharing, solving GPU allocation and communication across them, and providing data management tools. Mind you, with all its functionality, the toolkit is also incredibly complex. At the time of writing this, it has no sign of documentation yet… So, for now, MAMMOTH might be raw, but I hope it will mature into a useful tool for the research and production of multilingual neural MT and not another extinct elephantid.
One more tool worth highlighting is OpusCleaner, demonstrated by my friend and study mate, Nikolay Bogoychev from the University of Edinburgh, and developed with his colleagues Graeme Nail and Jelmer van der Linde. OpusCleaner is a training data cleaner for MT and language models. Unlike other data cleaning tools, OpusCleaner specifically aims to lessen the learning curve for training data cleaning by providing a graphical user interface for downloading datasets, configuring various filters, and providing immediate feedback on how applying these filters affects your data. This way, the daunting task of configuring and chaining data-cleansing workflows becomes intuitive and can be handed over to language experts without them needing a computer science background. Another cool thing about OpusCleaner is that it can be chained together with another tool by the same group, OpusTrainer, which handles data scheduling and sampling, thus reducing the workload and chance of human error in MT model training. Both tools are well documented and have an active developer community on their GitHub.
Overall, the 16th MT Marathon was a great success – there was a wide variety of lectures covering everything from the foundations of MT to opinion pieces and practical recommendations for novel approaches and tools. MT Marathon also offered a space for the exchange of ideas with colleagues from academia and industry. Above all, the event served as a platform for popularising incremental research so often overlooked by major academic conferences, but ever so important for mature, end-user-facing fields like MT.
