ktransformers KTransformers v0.7.2

KTransformers v0.7.2: DeepSeek-V4-Flash LoRA Fine-Tuning

KTransformers v0.7.2 adds LoRA fine-tuning for the April DeepSeek-V4-Flash checkpoint. Routed experts remain in native MXFP4 on CPU, while attention and other non-expert modules run on GPU.

🚀 Highlights

• Native MXFP4 routed-expert LoRA forward and backward, without expanding the frozen expert weights to BF16.
• LLaMA-Factory SFT integration with checkpoint saving and SGLang adapter inference.
• Python 3.11 and 3.12 Linux x86-64 wheels.

📦 Installation

For training:

python -m pip install 'ktransformers[sft]==0.7.2'

For inference:

python -m pip install 'ktransformers[sglang]==0.7.2'

🛠️ Main-Branch Updates

• KTransformers-FineTune preprint v1.1 upload and update our paper to the latest.
• CPU BF16 gradient-norm reduction adds a native sum-of-squares primitive for KT-owned gradients without materializing an FP32 tensor, which reduces the grad clip overhead of full fine-tuning.

🌟 Contributors

Thanks to all human code and documentation contributors since v0.7.1: @JimmyPeilinLi, @yyj6666667, @05yuki, @losewayy, @Dashener2, @pratikgx, @MohammadHijjawi97, and @Anai-Guo. Thanks also to everyone who reviewed and tested these changes.