Hi, thank you for releasing the DreamZero code.
I am studying training-free caching methods that exploit temporal redundancy during diffusion inference, and I had two questions about the DiT caching introduced in the System-level Optimizations section.
From my understanding, DreamZero measures the cosine similarity between velocity predictions from consecutive diffusion steps and skips DiT forward passes when the similarity exceeds a predefined threshold.
- How were the caching threshold and related settings determined?
The paper reports that DiT caching reduces the average number of DiT forward steps from 16 to around 4.
Were the cosine-similarity threshold and caching window chosen based mainly on the trade-off between downstream task performance and inference efficiency?
Or did you separately analyze the velocity trajectory during inference, such as the cosine similarity between consecutive velocity predictions across diffusion steps, and determine the caching settings based on those observations?
- Why are consecutive velocity predictions so directionally consistent in DreamZero?
In some flow-matching-based text-to-image and text-to-video models I examined, consecutive velocity predictions did not show similarly strong directional consistency.
Do you think this behavior is related to properties specific to WAMs or action-conditioned generation, or is it something you would generally expect in flow-matching-based WAMs?
If you have observed similar behavior in other World Action Models, I would also greatly appreciate any related observations or references.
Thank you for your time and for making the project publicly available.
Hi, thank you for releasing the DreamZero code.
I am studying training-free caching methods that exploit temporal redundancy during diffusion inference, and I had two questions about the DiT caching introduced in the System-level Optimizations section.
From my understanding, DreamZero measures the cosine similarity between velocity predictions from consecutive diffusion steps and skips DiT forward passes when the similarity exceeds a predefined threshold.
The paper reports that DiT caching reduces the average number of DiT forward steps from 16 to around 4.
Were the cosine-similarity threshold and caching window chosen based mainly on the trade-off between downstream task performance and inference efficiency?
Or did you separately analyze the velocity trajectory during inference, such as the cosine similarity between consecutive velocity predictions across diffusion steps, and determine the caching settings based on those observations?
In some flow-matching-based text-to-image and text-to-video models I examined, consecutive velocity predictions did not show similarly strong directional consistency.
Do you think this behavior is related to properties specific to WAMs or action-conditioned generation, or is it something you would generally expect in flow-matching-based WAMs?
If you have observed similar behavior in other World Action Models, I would also greatly appreciate any related observations or references.
Thank you for your time and for making the project publicly available.