FLUX.1 dev Controlnet Union EU AI Act Compliance Profile

FLUX.1-dev-Controlnet-Union

Release

• [2024/08/26] 🔥 Release FLUX.1-dev-ControlNet-Union-Pro. Please install from the source before the next release. We have supported CN-Union and Multi-ControlNets via this PR.

• [2024/08/20] Release the beta version.

• [2024/08/14] Release the alpha version.

Checkpoint

The training of union controlnet requires a significant amount of computational power. The current release is the first beta version checkpoint that maybe not been fully trained. The fully trainedbeta version is in the training process. We have conducted ablation studies that have demonstrated the validity of the code. The open-source release of the first beta version is solely to facilitate the rapid growth of the open-source community and the Flux ecosystem; it is common to encounter bad cases (please accept my apologies). It is worth noting that we have found that even a fully trained Union model may not perform as well as specialized models, such as pose control. However, as training progresses, the performance of the Union model will continue to approach that of specialized models.

Control Mode

| Control Mode | Description | Current Model Validity | |:------------:|:-----------:|:-----------:| |0|canny|🟢high| |1|tile|🟢high| |2|depth|🟢high| |3|blur|🟢high| |4|pose|🟢high| |5|gray|🔴low| |6|lq|🟢high|

Inference

import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetPipeline, FluxControlNetModel

base_model = 'black-forest-labs/FLUX.1-dev'
controlnet_model = 'InstantX/FLUX.1-dev-Controlnet-Union'

controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16)
pipe = FluxControlNetPipeline.from_pretrained(base_model, controlnet=contro

Your obligations as a FLUX.1 dev Controlnet Union deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library

Base Model