A Mathematical Framework for Transformer Circuits (2021)

 (transformer-circuits.pub)

61 points | by Bluestein 6 hours ago

4 comments

  • myself248 5 hours ago
    Nothing about B-H curves or magnetization current, must be that other kind of transformer, in that other kind of circuit.
    [-]
    • greenbit 2 hours ago
      Someone should step up and rectify that, there's a lot of potential for confusion.
      [-]
      • ziofill 2 hours ago
        This comment is hysteresical
        [-]
        • Bluestein 2 hours ago
          I find it polarizing.-
          [-]
          • thatspartan 1 hour ago
            It can be quite eddyfying
            [-]
            • pooyamo 22 minutes ago
              A harmonious comment about an impedance mismatch quickly spiraled into a galvanizing flow
  • _jayhack_ 2 hours ago
    Shocking to me how little interest the general public has in mechinterp given the alien capabilities demonstrated by LLMs - this and the subsequent transformer-circuits.pub publications will be seen as classic, foundational work in a few years
  • amelius 5 hours ago
    > A previous project, the Distill Circuits thread, has attempted to reverse engineer vision models, but so far there hasn’t been a comparable project for transformers or language models.

    How successful was that?

    [-]
  • oofbey 3 hours ago
    I’ve tried reading this many times. But it’s sooooo long. Is it worth it?
    [-]