T3D: Trajectory Self-Distillation via Direct Discriminative Optimization for Efficient Diffusion Language Models
Tunyu Zhang*, Xinxi Zhang*, Ligong Han, Haizhou Shi, Xiaoxiao He, Zhuowei Li, Hao Wang, Kai Xu, Akash Srivastava, Hao Wang, Vladimir Pavlovic, Dimitris Metaxas
Paper | Code | Slides

• T3D is a training framework for efficient diffusion language models via trajectory self-distillation.
• T3D uses Direct Discriminative Optimization (DDO) to replace mode-covering objectives with a mode-seeking training signal.
• The framework enables aggressive few-step generation while preserving full-step diffusion capabilities and reasoning performance.

TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
Tunyu Zhang*, Haizhou Shi*, Yibin Wang, Hengyi Wang, Xiaoxiao He, Zhuowei Li, Haoxian Chen, Ligong Han, Kai Xu, Huan Zhang, Dimitris Metaxas, Hao Wang
Paper | Code

• We propose TokUR, a framework for token-level uncertainty estimation tailored for LLM reasoning.
• TokUR introduces a low-rank stochastic perturbation mechanism to approximate predictive distributions efficiently.
• The framework enables more reliable multi-step reasoning, and provides uncertainty-aware signals for downstream tasks.

Multimodal needle in a haystack: Benchmarking long-context capability of multimodal large language models
Hengyi Wang, Haizhou Shi, Shiwei Tan, Weiyi Qin, Wenyuan Wang, Tunyu Zhang, Akshay Nambi, Tanuja Ganu, Hao Wang

Paper | Code

• MMNeedle provides a systematic evaluation framework for long-context multimodal understanding.
• It enables controlled benchmarking of retrieval and reasoning over large visual contexts, and reveals robustness challenges in current multimodal LLMs.