Official implementation of EchoCache, a training-free, energy-guided cross-modal caching framework for efficient audio-driven video generation.
2026/08🚀 The code of EchoCache is released.2026/08📄 The paper is available on arXiv.2026/08🎉 EchoCache is accepted by ACM Multimedia 2026.
Audio-driven video generation requires repeated denoising over the full video latent sequence, resulting in substantial inference latency. Existing caching methods mainly exploit redundancy in visual features and often overlook the highly non-uniform temporal importance of audio conditions.
EchoCache treats caching as a cross-modal computation-allocation problem. It converts temporal dynamics in audio signals into saliency cues that guide fine-grained cache updates across diffusion timesteps and video latents. EchoCache accelerates inference without additional training or modifications to the original model weights.
- Cross-Modal Saliency Anchors: Model the time-frequency energy distribution of audio signals to estimate the relative importance of temporal clips.
- Energy-Guided Step-Latent Cache Scheduling: Jointly controls cache reuse across diffusion timesteps and video latents, prioritizing recomputation for salient clips while reusing cached features for relatively stable clips.
- Memory-Efficient Cache Management: Stores reference features together with quantized residuals to reduce the memory overhead introduced by fine-grained caching.
- Training-Free Integration: Works as a plug-and-play acceleration layer for existing audio-driven video diffusion models.
- Fine-grained, audio-guided computation allocation instead of uniform feature reuse.
- Joint timestep-level and latent-level adaptive cache scheduling.
- Quantized residual cache management for improved memory efficiency.
- Support for Wan2.2-S2V and LongCat-Avatar.
- Up to 2.46× inference speedup on Wan2.2-S2V over the EMTD benchmark while preserving generation quality and audio-visual consistency.
| Model | Original repository | EchoCache implementation |
|---|---|---|
| Wan2.2-S2V | Wan2.2 | Wan2.2_echocache/ |
| LongCat-Avatar | LongCat-Video | LongCat-Video_echocache/ |
Important
EchoCache does not redistribute or modify the original model weights. Download the pretrained checkpoints from the corresponding official repositories and follow their respective licenses and usage requirements.
git clone https://github.com/IF-LAB-PKU/EchoCache.git
cd EchoCacheconda create -n echocache python=3.10 -y
conda activate echocache
pip install -r requirements.txtMake sure that FFmpeg is installed and available from the command line:
ffmpeg -versionSome model backbones may require additional PyTorch or CUDA extensions. Refer to the original model repositories and the instructions in the corresponding implementation directories:
Wan2.2_echocache/LongCat-Video_echocache/
EchoCache requires the original pretrained weights of each supported backbone.
- Download the Wan2.2-S2V weights by following the instructions in the Wan2.2 repository.
- Download the LongCat-Avatar weights by following the instructions in the LongCat-Video repository.
- Set the checkpoint paths in the corresponding launch scripts or configuration files before running inference.
The checkpoint formats and licenses remain unchanged from the original projects.
cd Wan2.2_echocache
bash run_s2v_echocache.shConfigure the input paths and EchoCache parameters in run_s2v_echocache.sh. The main options include:
| Option | Description |
|---|---|
--audio |
Path to the input audio file, such as .wav or .mp3. |
--image |
Path to the reference image, such as .jpg or .png. |
--selected_ratio |
Ratio of temporal latents selected for recomputation. |
--quantize_ratio |
Similarity threshold used to control cache quantization. |
--need_ffn |
Whether to perform complete FFN computation. |
cd LongCat-Video_echocache
bash run_echocache.shConfigure the audio input, reference image or video, pretrained checkpoint, output path, and caching parameters in run_echocache.sh before execution.
The principal EchoCache hyperparameters control:
- the sensitivity of the cross-modal saliency anchors;
- the ratio of temporal latents selected for recomputation;
- the timestep-level cache reuse schedule;
- the similarity threshold for residual quantization;
- whether complete FFN computation is enabled.
The recommended settings and parameter analyses are provided in the paper and supplementary material. When adapting EchoCache to a new backbone or dataset, start from the default configuration supplied for the closest supported model.
EchoCache/
├── Wan2.2_echocache/
│ └── run_s2v_echocache.sh
├── LongCat-Video_echocache/
│ └── run_echocache.sh
├── requirements.txt
└── README.md
Given an audio-conditioned video diffusion model, EchoCache first extracts the time-frequency energy distribution of the input audio and constructs cross-modal saliency anchors for different temporal clips. These anchors guide a step-latent cache scheduler that dynamically determines which latent clips should be recomputed and which can reuse historical features. To control the resulting storage overhead, EchoCache maintains reference caches and stores compact quantized residuals for subsequent updates.
This design aligns computation and storage decisions with the temporal dynamics of the audio condition, improving the latency-quality trade-off of audio-driven video generation.
We sincerely thank the teams behind Wan2.2 and LongCat-Video for releasing their models and code.
If you find EchoCache useful in your research, please cite our paper:
@inproceedings{chen2026echocache,
author = {Jiayu Chen and Xiaoyu Wu and Rongshan Gao and Maoliang Li
and Zihao Zheng and Xinhao Sun and Hailong Zou
and Guojie Luo and Xiang Chen},
title = {{EchoCache}: Energy-Guided Cross-Modal Caching for Efficient
Audio-Driven Video Generation},
booktitle = {Proceedings of the 34th ACM International Conference on Multimedia(MM'26)},
year = {2026}
}For questions or collaboration, please open an issue in this repository or contact Jiayu Chen.