Steering Video Diffusion Transformers
with Massive Activations

Xianhang Cheng1, Yujian Zheng1, Zhenyu Xie1, Tingting Liao1, Hao Li1,2
1MBZUAI    2Pinscreen

Abstract

Despite rapid progress in video diffusion transformers, how their internal model signals can be leveraged with minimal overhead to enhance video generation quality remains underexplored. In this work, we study the role of Massive Activations (MAs), which are rare, high-magnitude hidden state spikes in video diffusion transformers. We observed that MAs emerge consistently across all visual tokens, with a clear magnitude hierarchy: first-frame tokens exhibit the largest MA magnitudes, latent-frame boundary tokens (the head and tail portions of each temporal chunk in the latent space) show elevated but slightly lower MA magnitudes than the first frame, and interior tokens within each latent frame remain elevated, yet are comparatively moderate in magnitude. This structured pattern suggests that the model implicitly prioritizes token positions aligned with the temporal chunking in the latent space. Based on this observation, we propose Structured Activation Steering (STAS), a training-free self-guidance-like method that steers MA values at first-frame and boundary tokens toward a scaled global maximum reference magnitude. STAS achieves consistent improvements in terms of video quality and temporal coherence across different text-to-video models, while introducing negligible computational overhead.


Results

Side-by-side video comparisons: Baseline (left) vs. +Ours (right).

Wan2.1-1.3B


CogVideoX-5B


Wan2.2-5B


BibTeX

Please consider citing our paper if you find it useful in your research.

@article{cheng2026stas, title={Steering Video Diffusion Transformers with Massive Activations}, author={Cheng, Xianhang and Zheng, Yujian and Xie, Zhenyu and Liao, Tingting and Li, Hao}, journal={arXiv preprint arXiv:2603.17825}, year={2026} }