It would be great if this could be combined with quantization-aware finetuning. In my experience, Qwen3.6-27B has much fewer repetitions at Q6 quantization level as compared to e.g. Q4, but that leaves little space for context on my 24GB RTX 3090.
They are orthogonal; preference optimization like RLHF can be done on the base model which can later be quantized, or it could be done on a new LoRA that is then converted to QLoRA.
this is pretty cool. i think part of the root cause is current rlhf post training design around confidence and optics rather than cooperative transparent honesty. though its kinda an expensive hypothesis to dig into as a private individual
Most of the models where people are concerned about don't do this when unquantized, so I doubt it's much about the metapolitics imposed in reinforcement training.
2 comments