NitroFusion2

ChenDY/NitroFusion2

作者
ChenDY
任务
图像生成
下载量
874
点赞数
15
趋势分
15
许可证
apache-2.0
收录时间
2026/11/7

license: apache-2.0 language:

  • en pipeline_tag: text-to-image library_name: diffusers base_model: Tongyi-MAI/Z-Image-Turbo tags:
  • text-to-image
  • diffusion-distillation
  • adversarial-distillation
  • one-step
  • few-step
  • z-image-turbo
  • gguf
  • comfyui
  • nitrofusion2

NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers

GitHub Demo Project Page arXiv

🔥 News

✨ NitroFusion2

NitroFusion2 is a 1–2 step text-to-image model built on Z-Image-Turbo. It moves the adversarial loss onto the text stream of the MMDiT. At one step it renders sharp text and fine detail.

ComfyUI

  • 1 step: workflows/nitrofusion2_1step.json
  • 2 steps: workflows/nitrofusion2_2step.json

For two steps, we recommend our de-distillation LoRA, which takes NitroFusion2 back to Z-Image-Turbo, to eliminate some artifacts. The workflow applies it at scale 0.5 on the first step, then runs NitroFusion2 alone on the second step.

We also release GGUF tiers from nf2-Q2_K_4XS.gguf (2.0 GB) to nf2-Q8_0.gguf (7.2 GB).

GGUF workflows: 1 step, 2 steps.

Diffusers

Run inference.py (1–4 steps):

python inference.py --prompt "a cat holding a sign that says hello world" --steps 2 --out cat.png

Or in Python:

import torch
from diffusers import FlowMatchEulerDiscreteScheduler, ZImagePipeline, ZImageTransformer2DModel

transformer = ZImageTransformer2DModel.from_pretrained(
    "ChenDY/NitroFusion2",
    subfolder="transformer",
    torch_dtype=torch.bfloat16,
)
pipe = ZImagePipeline.from_pretrained(
    "Tongyi-MAI/Z-Image-Turbo",
    transformer=transformer,
    torch_dtype=torch.bfloat16,
)
pipe.load_lora_weights("ChenDY/NitroFusion2", weight_name="nf2-dedistill-lora.safetensors", adapter_name="dedistill")
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, shift=6.0)
pipe.enable_model_cpu_offload()

pipe.set_adapters("dedistill", 0.5)
def lora_off(pipe, step, timestep, callback_kwargs):
    pipe.set_adapters("dedistill", 0.0)
    return callback_kwargs

prompt = "Latina female with thick wavy hair, harbor boats and pastel houses behind. Breezy seaside light, warm tones, cinematic close-up."

image = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    num_inference_steps=2,
    sigmas=[1.0, 0.5],
    guidance_scale=0.0,
    generator=torch.Generator("cuda").manual_seed(42),
    callback_on_step_end=lora_off,
).images[0]

image.save("example.png")

Method

If this paper has one message, it is this: in an MMDiT, the text stream is no longer only conditioning. Once it reads the image, it becomes a discriminative representation, opening a new direction for adversarial diffusion distillation and allowing that discrimination to be steered toward specific aspects.

In an MMDiT, text and image tokens move through the same attention layers, so by the middle of the network the text tokens have already read the image. They hold a compact summary of what was generated, organised by what the prompt asked for. NitroFusion2 puts an adversarial loss there, on that summary, alongside the conventional one on the image features. Because the summary stays tied to the text condition, changing the condition changes what it reports — so one frozen discriminator can be asked a different question simply by swapping the cue. A typography cue makes it attend to rendered text, an object cue to the things named, a style cue to how the image looks. One backbone, many critics, at no extra cost.

Citation

If you find NitroFusion2 useful or relevant to your research, please kindly cite our paper:

@article{chen2026nitrofusion2,
    title={NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers},
    author={Chen, Dar-Yen and Song, Yi-Zhe},
    journal={arXiv preprint arXiv:2611.05648},
    year={2026}
}

Please also consider citing NitroFusion:

@inproceedings{chen2025nitrofusion,
    title={NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training},
    author={Dar-Yen Chen and Hmrishav Bandyopadhyay and Kai Zou and Yi-Zhe Song},
    booktitle={CVPR},
    year={2025}
}

内容来源:Hugging Face 模型卡

在 Hugging Face 查看