Initial commit

Co-authored-by: jindidi <jindidi@users.noreply.huggingface.co>
Co-authored-by: kuan-wang <kuan-wang@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,53 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets[[:space:]]/example1-eigen-banana.png filter=lfs diff=lfs merge=lfs -text
+assets[[:space:]]/example1-input.png filter=lfs diff=lfs merge=lfs -text
+assets[[:space:]]/example1-qwen-image-edit.png filter=lfs diff=lfs merge=lfs -text
+assets/example1-eigen-banana.png filter=lfs diff=lfs merge=lfs -text
+assets/example1-input.png filter=lfs diff=lfs merge=lfs -text
+assets/example1-qwen-image-edit.png filter=lfs diff=lfs merge=lfs -text
+assets/example2-eigen-banana.png filter=lfs diff=lfs merge=lfs -text
+assets/example2-input.png filter=lfs diff=lfs merge=lfs -text
+assets/example2-qwen-image-edit.png filter=lfs diff=lfs merge=lfs -text
+assets/example3-eigen-banana.png filter=lfs diff=lfs merge=lfs -text
+assets/example3-input.png filter=lfs diff=lfs merge=lfs -text
+assets/example3-qwen-image-edit.png filter=lfs diff=lfs merge=lfs -text
+assets/example4-eigen-banana.png filter=lfs diff=lfs merge=lfs -text
+assets/example4-input.png filter=lfs diff=lfs merge=lfs -text
+assets/example4-qwen-image-edit.png filter=lfs diff=lfs merge=lfs -text
+assets/example5-eigen-banana.png filter=lfs diff=lfs merge=lfs -text
+assets/example5-input.png filter=lfs diff=lfs merge=lfs -text
+assets/example5-qwen-image-edit.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,507 @@
+---
+license: apache-2.0
+tags:
+- text-to-image
+- image-editing
+- lora
+- qwen-image
+- diffusion
+library_name: diffsynth
+base_model:
+- Qwen/Qwen-Image-Edit
+---
+
+# Eigen-Banana-Qwen-Image-Edit: Fast Image Editing with Qwen-Image-Edit LoRA
+
+⚡ [**Lightning Demo Website**](https://app.eigenai.com/eigen-banana-qwen-image-edit.html) / 📄 [**Blog Post**](https://www.eigenai.com/blog/2025-10-30-eigen-banana.html)
+
+**Eigen-Banana-Qwen-Image-Edit** is a LoRA (Low-Rank Adaptation) checkpoint for the Qwen-Image-Edit model, optimized for fast, high-quality image editing with text prompts. This model enables efficient text-guided image transformations with reduced inference steps while maintaining excellent quality.
+
+Trained on the **[Pico-Banana-400K](https://github.com/apple/pico-banana-400k)** dataset from Apple—a large-scale collection of ~400K text–image–edit triplets covering 35 edit operations across diverse semantic categories—Eigen-Banana-Qwen-Image-Edit excels at a wide range of editing tasks from object manipulation to stylistic transformations.
+
+## Model Details
+
+- **Base Model**: Qwen/Qwen-Image-Edit
+- **Model Type**: LoRA Fine-tuned Diffusion Transformer
+- **Training Dataset**: [Pico-Banana-400K](https://github.com/apple/pico-banana-400k)
+- **Training Method**: EigenTrain (LoRA fine-tuning)
+- **Format**: FP16 SafeTensors
+- **License**: Apache 2.0
+- **Use Cases**: Text-guided image editing, style transfer, object modification, scene transformation
+
+## Features
+
+✨ **Fast Inference**: Optimized for quick generation with distilled knowledge  
+🎨 **High Quality**: Maintains excellent visual quality with fewer steps  
+🌐 **Multilingual**: Supports both English and Chinese prompts  
+⚡ **Efficient**: Lightweight LoRA weights for easy deployment  
+
+## Training Dataset
+
+This model was trained on **[Pico-Banana-400K](https://github.com/apple/pico-banana-400k)**, a large-scale dataset of ~400K text–image–edit triplets designed for text-guided image editing research.
+
+### Dataset Highlights
+
+- **~257K single-turn text–image–edit triplets** for supervised fine-tuning
+- **35 edit operations** across **8 semantic categories**:
+  - Object-Level Semantic (35%): Add, remove, replace, or relocate objects
+  - Scene Composition & Multi-Subject (20%): Contextual transformations
+  - Human-Centric (18%): Clothing, expression, appearance edits
+  - Stylistic (10%): Domain and artistic style transfer
+  - Text & Symbol (8%): Edits involving visible text or signs
+  - Pixel & Photometric (5%): Brightness, contrast, tonal adjustments
+  - Scale & Perspective (2%): Zoom, viewpoint changes
+  - Spatial/Layout (2%): Outpainting, composition extension
+
+- **Source Images**: [Open Images Dataset](https://storage.googleapis.com/openimages/web/index.html)
+- **Instruction Generation**: Gemini-2.5-Flash for natural-language editing prompts
+- **Quality Control**: Automated self-evaluation using Gemini-2.5-Pro
+
+### Training Methodology
+
+The model was fine-tuned using **EigenTrain**, a training platform that unifies SFT, offline RL, and online RL for training text LLMs and VLMs, and includes first-class workflows for multimodal image/video generation. Here, we used EigenTrain to do LoRA fine-tuning on Qwen-Image-Edit. Key training parameters:
+
+- **LoRA Rank**: 32
+- **Target Modules**: `to_q`, `to_k`, `to_v`, `add_q_proj`, `add_k_proj`, `add_v_proj`, `to_out.0`, `to_add_out`, `img_mlp.net.2`, `img_mod.1`, `txt_mlp.net.2`, `txt_mod.1`
+- **Learning Rate**: 1e-4
+- **Training Data**: ~257K high-quality text-image-edit triplets
+- **Precision**: FP16 for efficient deployment
+
+This combination of high-quality training data and efficient LoRA adaptation enables fast, accurate image editing while maintaining the base model's strong capabilities.
+
+## Demo Images
+
+We present several examples to qualitatively compare the original qwen-image-edit and our eigen-banana-qwen-image-edit. 
+
+### Example 1 – Add a new object to the scene
+
+<!-- Input (moderate width, centered) -->
+<p align="center">
+  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example1-input.png"
+       alt="example1-input"
+       width="360">
+</p>
+
+**Prompt:** Integrate a minimalist, dark-toned, rectangular gallery bench into the mid-ground, positioned slightly to the right of the central pillar and facing the right wall, ensuring its texture, lighting, and subtle shadows are consistent with the existing black and white aesthetic and diffused ambient light of the art gallery.
+
+**Outputs**
+
+<!-- Two-up grid with captions (works on HF + GitHub) -->
+<table>
+  <tr>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example1-qwen-image-edit.png"
+           alt="example1-qwen-image-edit"
+           width="100%"><br/>
+      <h5><b>Qwen-Image-Edit</b></h5>
+    </td>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example1-eigen-banana.png"
+           alt="example1-eigen-banana"
+           width="100%"><br/>
+      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
+    </td>
+  </tr>
+</table>
+
+
+### Example 2 – Add a film grain/filter
+
+<!-- Input (moderate width, centered) -->
+<p align="center">
+  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example2-input.png"
+       alt="example2-input"
+       width="360">
+</p>
+
+**Prompt:** Apply a vintage film aesthetic to the image, featuring a subtle desaturation of colors with a warm, golden-hour tone, introduce a fine and natural-looking film grain across the entire scene, gently reduce overall contrast for a softer appearance, and add a very faint, dark vignette to the edges to mimic an aged photographic print.
+
+**Outputs**
+
+<!-- Two-up grid with captions (works on HF + GitHub) -->
+<table>
+  <tr>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example2-qwen-image-edit.png"
+           alt="example2-qwen-image-edit"
+           width="100%"><br/>
+      <h5><b>Qwen-Image-Edit</b></h5>
+    </td>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example2-eigen-banana.png"
+           alt="example2-eigen-banana"
+           width="100%"><br/>
+      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
+    </td>
+  </tr>
+</table>
+
+
+### Example 3 – Add a new text
+
+<!-- Input (moderate width, centered) -->
+<p align="center">
+  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example3-input.png"
+       alt="example3-input"
+       width="360">
+</p>
+
+**Prompt:** Add the text "CHAMPION" in a bold, sans-serif font, horizontally aligned below the existing "GLASGOW" text on the race bib of the runner wearing number 454 (yellow singlet), ensuring the text color, lighting, and subtle fabric distortion match the existing elements on the bib.
+
+**Outputs**
+
+<!-- Two-up grid with captions (works on HF + GitHub) -->
+<table>
+  <tr>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example3-qwen-image-edit.png"
+           alt="example3-qwen-image-edit"
+           width="100%"><br/>
+      <h5><b>Qwen-Image-Edit</b></h5>
+    </td>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example3-eigen-banana.png"
+           alt="example3-eigen-banana"
+           width="100%"><br/>
+      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
+    </td>
+  </tr>
+</table>
+
+
+### Example 4 – Add a new scene/background 
+
+<!-- Input (moderate width, centered) -->
+<p align="center">
+  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example4-input.png"
+       alt="example4-input"
+       width="360">
+</p>
+
+**Prompt:** Replace the current plain wall background with a sophisticated, softly lit indoor event space, featuring warm golden ambient lighting, elegant architectural details such as decorative panels or subtle artwork, and a slightly blurred depth of field to keep the focus on the subjects while ensuring the new background's rich, muted tones complement their attire.
+
+**Outputs**
+
+<!-- Two-up grid with captions (works on HF + GitHub) -->
+<table>
+  <tr>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example4-qwen-image-edit.png"
+           alt="example4-qwen-image-edit"
+           width="100%"><br/>
+      <h5><b>Qwen-Image-Edit</b></h5>
+    </td>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example4-eigen-banana.png"
+           alt="example4-eigen-banana"
+           width="100%"><br/>
+      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
+    </td>
+  </tr>
+</table>
+
+
+### Example 5 – Modify expressions 
+
+<!-- Input (moderate width, centered) -->
+<p align="center">
+  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example5-input.png"
+       alt="example5-input"
+       width="360">
+</p>
+
+**Prompt:** Adjust the subject's facial expression to a subtle, closed-mouth smile, ensuring natural skin folds and realistic lighting on the face, while maintaining the existing head posture and integrating seamlessly with the overall image context.
+
+**Outputs**
+
+<!-- Two-up grid with captions (works on HF + GitHub) -->
+<table>
+  <tr>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example5-qwen-image-edit.png"
+           alt="example5-qwen-image-edit"
+           width="100%"><br/>
+      <h5><b>Qwen-Image-Edit</b></h5>
+    </td>
+    <td align="center" width="50%">
+      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example5-eigen-banana.png"
+           alt="example5-eigen-banana"
+           width="100%"><br/>
+      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
+    </td>
+  </tr>
+</table>
+
+
+## Installation
+
+### Requirements
+
+- Python 3.8+
+- CUDA-capable GPU (recommended)
+- 8GB+ VRAM
+
+### Setup
+
+1. Clone the DiffSynth-Studio repository:
+
+```bash
+git clone https://github.com/modelscope/DiffSynth-Studio.git
+cd DiffSynth-Studio
+```
+
+2. Install the package:
+
+```bash
+pip install -e .
+```
+
+3. Install additional dependencies:
+
+```bash
+pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
+```
+
+## Usage
+
+### Basic Image Editing
+
+```python
+from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
+import torch
+
+# Initialize the pipeline
+pipe = QwenImagePipeline.from_pretrained(
+    torch_dtype=torch.bfloat16,
+    device="cuda",
+    model_configs=[
+        ModelConfig(
+            model_id="Qwen/Qwen-Image-Edit",
+            origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"
+        ),
+        ModelConfig(
+            model_id="Qwen/Qwen-Image",
+            origin_file_pattern="text_encoder/model*.safetensors"
+        ),
+        ModelConfig(
+            model_id="Qwen/Qwen-Image",
+            origin_file_pattern="vae/diffusion_pytorch_model.safetensors"
+        ),
+    ],
+    processor_config=ModelConfig(
+        model_id="Qwen/Qwen-Image-Edit",
+        origin_file_pattern="processor/"
+    ),
+)
+
+# Load the Eigen-Banana-Qwen-Image-Edit LoRA
+pipe.load_lora(pipe.dit, "eigen-ai-labs/eigen-banana-qwen-image-edit/eigen-banana-qwen-image-edit-fp16-lora.safetensors")
+
+# Generate an initial image
+prompt = "A beautiful portrait, underwater girl, blue dress flowing, hair drifting, light penetrating, bubbles surrounding, serene face, exquisite details, dreamy and aesthetic."
+input_image = pipe(
+    prompt=prompt,
+    seed=0,
+    num_inference_steps=40,
+    height=1328,
+    width=1024
+)
+input_image.save("original.jpg")
+
+# Edit the image
+edit_prompt = "Change the dress to pink"
+edited_image = pipe(
+    edit_prompt,
+    edit_image=input_image,
+    seed=1,
+    num_inference_steps=40,
+    height=1328,
+    width=1024,
+    edit_image_auto_resize=True
+)
+edited_image.save("edited.jpg")
+```
+
+### Chinese Prompts Example
+
+```python
+# Generate initial image with Chinese prompt
+prompt = "精致肖像，水下少女，蓝裙飘逸，发丝轻扬，光影透澈，气泡环绕，面容恬静，细节精致，梦幻唯美。"
+input_image = pipe(
+    prompt=prompt,
+    seed=0,
+    num_inference_steps=40,
+    height=1328,
+    width=1024
+)
+input_image.save("image1.jpg")
+
+# Edit with Chinese prompt
+prompt = "将裙子改为粉色"
+edited_image = pipe(
+    prompt,
+    edit_image=input_image,
+    seed=1,
+    num_inference_steps=40,
+    height=1328,
+    width=1024,
+    edit_image_auto_resize=True
+)
+edited_image.save("image2.jpg")
+```
+
+### Advanced Usage
+
+#### Auto-Resize Options
+
+The `edit_image_auto_resize` parameter controls how input images are processed:
+
+```python
+# Auto-resize: maintains aspect ratio while matching 1024x1024 area
+edited_image = pipe(
+    prompt,
+    edit_image=input_image,
+    num_inference_steps=40,
+    height=1328,
+    width=1024,
+    edit_image_auto_resize=True  # Recommended for best results
+)
+
+# No resize: use original image size
+edited_image = pipe(
+    prompt,
+    edit_image=input_image,
+    num_inference_steps=40,
+    height=1328,
+    width=1024,
+    edit_image_auto_resize=False  # Use when input matches target size
+)
+```
+
+#### Inference Steps Optimization
+
+The model works well with reduced steps for faster generation:
+
+```python
+# High quality (slower)
+image = pipe(prompt, edit_image=input_image, num_inference_steps=40)
+
+# Balanced (recommended)
+image = pipe(prompt, edit_image=input_image, num_inference_steps=20)
+
+# Fast (may reduce quality slightly)
+image = pipe(prompt, edit_image=input_image, num_inference_steps=10)
+```
+
+## Example Prompts
+
+### English Prompts
+
+- "Transform this image into a cartoon style"
+- "Change the background to a sunset beach"
+- "Make it look like a watercolor painting"
+- "Add neon lights in cyberpunk style"
+- "Convert to black and white photograph"
+- "Change the sky to nighttime with stars"
+
+### Chinese Prompts
+
+- "将图片转换为卡通风格"
+- "把背景改成夕阳海滩"
+- "改成水彩画风格"
+- "添加赛博朋克风格的霓虹灯"
+- "转换为黑白照片"
+- "将天空改为夜晚星空"
+
+## Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `prompt` | str | - | Text description for image generation/editing |
+| `edit_image` | PIL.Image | None | Input image to edit (omit for text-to-image) |
+| `num_inference_steps` | int | 20 | Number of denoising steps |
+| `height` | int | 1024 | Output image height |
+| `width` | int | 1024 | Output image width |
+| `seed` | int | 0 | Random seed for reproducibility |
+| `edit_image_auto_resize` | bool | True | Auto-resize input to match target area |
+
+## Performance
+
+- **Inference Speed**: ~2-3 seconds per image (40 steps, RTX 4090)
+- **VRAM Usage**: ~6-8GB (bfloat16)
+- **Recommended Steps**: 20-40 for optimal quality
+
+## Limitations
+
+- Works best with natural images and common objects
+- May struggle with highly abstract or unusual concepts
+- Quality depends on prompt clarity and specificity
+- Very large images may require more VRAM
+
+## Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@software{eigen-banana-qwen-image-edit,
+  title={Eigen-Banana-Qwen-Image-Edit: Fast Image Editing LoRA for Qwen-Image-Edit},
+  author={Eigen AI Labs},
+  year={2025},
+  url={https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit}
+}
+```
+
+If you use the Pico-Banana-400K dataset, please also cite:
+
+```bibtex
+@misc{qian2025picobanana400k,
+  title={Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editing}, 
+  author={Yusu Qian and Eli Bocek-Rivele and Liangchen Song and Jialing Tong and Yinfei Yang and Jiasen Lu and Wenze Hu and Zhe Gan},
+  year={2025},
+  eprint={2510.19808},
+  archivePrefix={arXiv},
+  primaryClass={cs.CV},
+  url={https://arxiv.org/abs/2510.19808}
+}
+```
+
+## License
+
+This model is a derivative work of the Qwen-Image-Edit model, which is released under the Apache 2.0 License.
+
+The model was trained using the Pico-Banana-400K dataset, which is released under the Creative Commons Attribution–NonCommercial–NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License.
+
+License Details:
+
+- Apache 2.0 License applies to the parts of the model derived from the Qwen-Image-Edit model.
+
+- CC BY-NC-ND 4.0 License applies to the trained model due to the restrictions of the training dataset. Specifically, you may not:
+
+  - Use the model for commercial purposes.
+
+  - Create derivative works of the model.
+
+  - Distribute the model or dataset in a manner that violates the NoDerivatives condition.
+
+Restrictions:
+
+- The model may only be used for non-commercial purposes.
+
+- You may not modify, adapt, or build upon the model.
+
+- You may not distribute the model if the modifications to it would violate the NoDerivatives clause of the dataset’s license.
+
+## Acknowledgements
+
+- Trained on the [Pico-Banana-400K](https://github.com/apple/pico-banana-400k) dataset by Apple
+- Built on top of [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio)
+- Based on [Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) and [Qwen-Image-Edit](https://huggingface.co/Qwen/Qwen-Image-Edit)
+- Special thanks to the Apple ML team for releasing the high-quality Pico-Banana-400K dataset
+
+## Contact
+
+For questions, issues, or collaborations, please contact us at: https://www.eigenai.com/contact.
+
+---
+
+**Note**: This is a LoRA checkpoint and requires the base Qwen-Image-Edit model to function. The base models will be automatically downloaded from HuggingFace when you run the code.

eigen-banana-qwen-image-edit-fp16-lora.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34b9891926a444cf0ed62233790f58c4200d8bee55c401bb3f6dd8fde9241df1
+size 472047184