ID-Patch: Robust ID Association for Group Photo Personalization

Yimeng Zhang 1,2,*, Tiancheng Zhi1, Jing Liu1, Shen Sang1, Liming Jiang1, Qing Yan1, Sijia Liu2, Linjie Luo1
1ByteDance Inc., 2Michigan State University
( Code and model are coming soon!! )
*Work done during internship at ByteDance.
Paper Code arXiv
Interpolate start reference image.

Comparison with state-of-the-art multi-identity generation methods. From left to right: the condition inputs followed by results generated using OMG (with InstantID ), InstantFamily , and our proposed ID-Patch approach. Red dashed boxes highlight failures. OMG fails to preserve the hairstyles of the middle person, and creates artifacts for the right woman's hand, possibly because of the inconsistency between its generation stages . InstantFamily suffers from ID leakage, resulting the incorrect ID of the middle person. Our approach preserves the detailed identity of each person. In addition, our approach is 7 times faster than OMG and has less computational overhead than InstantFamily.

Abstract

The ability to synthesize personalized group photos and specify the positions of each identity offers immense creative potential. While such imagery can be visually appealing, it presents significant challenges for existing technologies. A persistent issue is identity (ID) leakage, where injected facial features interfere with one another, resulting in low face resemblance, incorrect positioning, and visual artifacts. Existing methods suffer from limitations such as the reliance on segmentation models, increased runtime, or a high probability of ID leakage. To address these challenges, we propose ID-Patch, a novel method that provides robust association between identities and 2D positions. Our approach generates an ID patch and ID embeddings from the same facial features: the ID patch is positioned on the conditional image for precise spatial control, while the ID embeddings integrate with text embeddings to ensure high resemblance. Experimental results demonstrate that ID-Patch surpasses baseline methods across metrics, such as face ID resemblance, ID-position association accuracy, and generation efficiency.

Pipeline of ID-Patch

Interpolate start reference image

Pose-Free Generation using ID-Patch

Interpolate start reference image

Plug-and-Play: Canny Edge

Interpolate start reference image
Combination of pose-free ID-Patch ControlNet and pretrained Canny edge ControlNet.

Pose-Conditioned Generation

Interpolate start reference image Interpolate start reference image Interpolate start reference image Interpolate start reference image Interpolate start reference image

BibTeX

@article{zhang2024idpatch,
        title={ID-Patch: Robust ID Association for Group Photo Personalization},
        author={Zhang, Yimeng and Zhi, Tiancheng and Liu, Jing and Sang, Shen and Jiang, Liming and Yan, Qing and Liu, Sijia and Luo, Linjie},
        journal={arXiv preprint arXiv:2411.13632},
        year={2024}
      }