Explanation of INT8 ConvRot (FP8 is no longer needed)

3 comments

• liuliu 1 hour ago
  One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding vector, and it is required now, so you cannot easily get the benefit for prefill only, while maintaining the same decoding performance.
• augment_me 1 hour ago
  Alternative title: "Our invention makes INT8 better on older consumer cards without FP8 support"
• peter_d_sherman 1 hour ago
  >"FP8, which has been used as the standard until now, was only supported by the RTX 40/50 series. In contrast,

  INT8 ConvRot

  is attracting attention because it can benefit from hardware support even on the RTX 20/30 series."

  Related:

  https://blog.comfy.org/p/minimax-h3-day-0-support-in-comfyui

  [-]
  • spottedmarley 1 hour ago
    Just read that before finding this. Makes me feel like we're all on the same brain-wavelength. Im actually in the middle of pulling and setting up H3 locally (int8 convrot)