Show HN: LLM Attention Visualization

 (ishamf.dev)

27 points | by ifz 1 hour ago

2 comments

  • sva_ 32 minutes ago
    I highly question this simplistic idea of high vector magnitude = high influence.
    [-]
    • smallmancontrov 13 minutes ago
      You get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.htm...
    • ifz 23 minutes ago
      I don't disagree with that. I did add an entire caveat paragraph there.

      To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.

  • colophontio 49 minutes ago
    [dead]