Patil/Krea-2-depth-controlnet · Hugging Face

Source: https://huggingface.co/Patil/Krea-2-depth-controlnet

https://huggingface.co/Patil/Krea-2-depth-controlnet#krea-2-depth-controlnet-loraKrea-2 Depth ControlNet-LoRA

Depth-conditioned generation forKrea-2. Give it any image and a prompt — it extracts the depth map with Depth-Anything-V2 and generates a new image with thesame 3D structureand composition, but whatever content and style you ask for.

• Trained onKrea-2-Raw, works on bothRawandKrea-2-Turbo(8-step)
• Single 862MB LoRA file (rank 64 + expanded input projection), base stays frozen
• Depth consistency (Pearson corr. of input depth vs. depth of generated image):0.98with no prompt,0.99with prompts

Each strip: init image → extracted depth → generated output.

https://huggingface.co/Patil/Krea-2-depth-controlnet#examplesExamples

https://huggingface.co/Patil/Krea-2-depth-controlnet#checkpointCheckpoint

https://huggingface.co/Patil/Krea-2-depth-controlnet#comfy-uiComfy UI

For comfy ui you follow the guide given here :https://github.com/facok/comfyui-krea2-controlnet

https://huggingface.co/Patil/Krea-2-depth-controlnet#setupSetup

git clone https://github.com/Tanmaypatil123/Krea-2-controlnet.git
cd Krea-2-controlnet
pip install -r requirements.txt

hf download Patil/Krea-2-depth-controlnet depth-control-lora.safetensors --local-dir .

https://huggingface.co/Patil/Krea-2-depth-controlnet#inferenceInference

# Turbo base — fast, recommended (8 steps, no CFG)
python inference.py photo.jpg -p "a futuristic spaceship interior, cinematic lighting" \
    --lora depth-control-lora.safetensors

# Raw base — undistilled (28-52 steps, CFG 3.5)
python inference.py photo.jpg -p "..." --lora depth-control-lora.safetensors \
    --base raw

# No prompt: the depth map is the only signal
python inference.py photo.jpg --lora depth-control-lora.safetensors --save-strip

# Weaker structure adherence (more creative freedom)
python inference.py photo.jpg -p "..." --lora depth-control-lora.safetensors --lora-scale 0.6

flagdefaultnotes\-p / \-\-prompt``""empty = depth-only generation\-\-base``turbo``turboorraw``\-\-steps8 turbo / 28 raw\-\-cfg0 turbo / 3.5 rawclassifier-free guidance\-\-mu1.15 turbo / auto rawtimestep shift\-\-lora\-scale1.0control-strength dial\-\-seed0\-\-save\-stripoffalso saves input|depth|output comparison

https://huggingface.co/Patil/Krea-2-depth-controlnet#python-apiPython API

from PIL import Image
from huggingface_hub import hf_hub_download
from pipeline import DepthLoRAPipeline

base = hf_hub_download("krea/Krea-2-Turbo", "turbo.safetensors")
pipe = DepthLoRAPipeline(base, "depth-control-lora.safetensors")

out, depth = pipe(Image.open("photo.jpg"),
                  prompt="a cozy cabin interior at dusk",
                  steps=8, cfg=0.0, mu=1.15, seed=0)
out.save("output.png")

https://huggingface.co/Patil/Krea-2-depth-controlnet#how-it-works-inference-pathHow it works (inference path)

1. The init image is resized to the nearest ~1MP aspect bucket and run throughDepth-Anything-V2-Large→ inverse depth map (near = white).
2. The depth map is encoded with the sameQwen-Image VAEthe model uses for images, so control lives in latent space.
3. At every denoising step, the depth latent isconcatenated channel-wiseto the noisy latent (each DiT token: 64 → 128 dims). The expanded input projection + rank-64 LoRA on all 28 blocks (both included in the checkpoint) steer generation to follow the depth structure.
4. Standard Krea-2 flow-matching Euler sampling otherwise — same recipe as BFL’s Flux.1-Depth-dev-lora.

https://huggingface.co/Patil/Krea-2-depth-controlnet#tips–limitationsTips & limitations

• Best inputs: photos / renders with real perspective. Flat 2D illustrations produce nearly-uniform depth maps, so control will be weak (garbage in, garbage out).
• Empty-prompt generation works (0.98 depth consistency) — useful for testing how much structure the control alone carries.
• \-\-lora\-scalebelow 1.0 relaxes structure adherence; above 1.0 tightens it at some quality cost.
• Krea-2-Raw generates up to ~1K resolution; outputs are capped at the ~1MP buckets.

https://huggingface.co/Patil/Krea-2-depth-controlnet#filesFiles

• inference\.py— CLI
• pipeline\.py— full pipeline: LoRA surgery, Qwen3-VL conditioner, VAE, depth estimator, flow sampler with control injection
• mmdit\.py— unmodified DiT definition from thekrea-2 repo

Model weights are subject to theKrea 2 community license. Training code will be released separately.

@Tanmaypatil79,@Shauray7,@edwixxxx