The specific practice of visualization of TEMPORAL Attention Features

[lyzcool]

Hello author, your excellent work has inspired me a lot. I have a question about how you visualize TEMPORAL Attention Features. If possible, could you please provide the code? Thank you very much!

[Gjmustc]

I also have the same doubt. The temporal attention visualization I have tried myself is a diagonal band without a clear outline of the video subject. Can you share the code for visualizing the attention map? It would be highly appreciated!

[wren93]

Hi, for each patch in each frame, we visualize the average attention score across all attention heads and all tokens (i.e. average
attention weights for all other tokens attending to the current token). So for temporal attention, this average score is computed across 16 tokens at the same spatial position along the temporal dimension. Sorry I no longer have the visualization code.

[Gjmustc]

Thank you for your timely reply! I also want to know why the spatial attention scores in the I2VGen-XL backbone is all low. Is there any other method to obtain a visualization result of the cross-attn or spatial-attn probability distributions?

[wren93]

It's not all low, it contains very high values at 1 or 2 positions (as shown by the white dots in the figure). You can take the cross attention and spatial attention weights and draw a box plot or histogram for visualization.