T$D^2$:Text-Driven Decoding for Mitigating Image-Induced Hallucination in Low-Resource Multimodal Translation
Published:
Please cite:
@INPROCEEDINGS{yichen2026,
author={Yi Chen, Mieradilijiang Maimaiti, Yu Pei, Jiaxin Wang, Shuoshuo Hou, Zhexin Li, Zhengyi Guo, and Wushour Silamu},
booktitle={The 22nd China Conference on Machine Translation and Multilingual Technology (CCMT, 2026)},
title={T$D^2$:Text-Driven Decoding for Mitigating Image-Induced Hallucination in Low-Resource Multimodal Translation},
year={2026},
volume={},
number={},
pages={},
keywords={Multimodal Machine Translation, Low-Resource Machine Translation, Vision-Language Models, Hallucination},
doi={},
}
Abstract
Large Vision-Language Models (LVLMs) have shown strong potential for multimodal machine translation by leveraging visual context. However, visual inputs can also introduce image-induced priors, leading to visually plausible but textually unfaithful translations, especially for low-resource target languages. We propose Text-Driven Decoding (T$D^2$), a simple, training-free method that calibrates next-token logits by contrasting those obtained from the full multimodal input with their image-only counterparts. By suppressing source-agnostic image-prior logits while preserving the full source-image context, T$D^2$ improves source-faithful translation without model retraining. Experiments on VG and Multi30K with five LVLM backbones spanning three model families show improvements over direct decoding, while comparisons on two non-Qwen backbones further demonstrate gains over VCD and ICD. Entity-level analyses further indicate reduced source-unsupported hallucinations in multiple settings. The corresponding source code is available at \footnote{\url{https://github.com/chenyi240226-oss/Text-Driven-Decoding}}.
[PDF] [Code] [Poster] [Slides] [Best Poster Award]
