Stable Diffusion ControlNet：Reference“垫图”功能，不炼丹也能保持同一人物

ControlNet推出Reference(参考)控制类型也有一段时间了，最近体验了一下，被他的强大的效果震惊到了，之前一直想生成同一主体系列图片，需要Lora炼丹，这个不仅需要电脑有一定的配置，还要处理大量素材图片耗时耗力，现在Reference却可以调调参数，测测数据，毫不费力的满足你需求！文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

下面来了解一下这个Reference模式，到底有多强大呢？文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

1、先看一下官方对Reference（参考）的介绍：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

Now we have a reference-only preprocessor that does not require any control models. It can guide the diffusion directly using images as references.文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

翻译：现在我们有一个 reference-only预处理，它不需要任何控制模型即可实现直接使用一张图片作为参考来引导扩散。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

2、再来看看Reference（参考）界面：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

预处理器：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

reference_adain：仅参考输入图，自适应实例规范文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

reference_adain+attn：仅参考输入图，自适应实例规范+Attention链接文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

reference_only：仅参考输入图，今天主要讲这个预处理器。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

控制权重：控制模型对生成图片的影响的程度。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

引导时机：控制模型介入的时间。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

引导终止时机：控制模型终止的时间。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

Style Fidelity(仅用于均衡模式)：风格保真度，文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

3、来个可爱的猫咪案例演示：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

在网上找一张背景干净，主体清晰的图片，这里找的一个猫，将图片改为512x512px尺寸，文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

放入controlnet，启用reference。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

选择预处理器：reference_only。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

style fidelity：设置为1。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

模型：realisticVisionV20。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

设置提示词：就一个词，一只行走的猫。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

正向：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

(best quality),((masterpiece)),(highres),original,extremely detailed 8K wallpaper,(an extremely delicate and beautiful),文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

a walking cat,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

反向：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

easynegative,ng_deepnegative_v1_75t,(worst quality:2),(low quality:2),(normal quality:2),lowres,bad anatomy,bad hands,normal quality,((monochrome)),((grayscale)),((watermark)),文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

其他的不用管，点击生成图片。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

可以看出与原素材底图作比较，除了一细节的差别，与原图匹配度高达80%，在降低随机性，提高抽卡中奖率上面还是有比较大的效果的，在生成足够多的样本后，我们可以挑选一些匹配度更高的图片。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

下面看下取消reference的效果：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

AI自由发挥，风格基本难以掌控。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

4、再来一个靓妹演示案例来测试，据说这样看的人会多些：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

模型：realisticVisionV20文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

正向：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

photo of woman,wispy blonde hair,perfect eyes,simple long knitted tight-fitting shift dress,(detailed face:1.2),(flat chest:1.1),slight smile,walking,outdoor,nature,beautiful,wild flowers,natural skin,dusk,orange clouds,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

8k uhd,high quality,ray tracing,(best quality),((masterpiece)),(highres),文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

反向：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

multiple breasts,(mutated hands and fingers:1.5 ),(long body :1.3),(mutation, poorly drawn :1.2),black-white,bad anatomy,liquid body,liquid tongue,disfigured,malformed,mutated,anatomical nonsense,text font ui,error,malformed hands,long neck,blurred,lowers,lowres,bad anatomy,bad proportions,bad shadow,uncoordinated body,unnatural body,fused breasts,bad breasts,huge breasts,poorly drawn breasts,extra breasts,liquid breasts,heavy breasts,missing breasts,huge haunch,huge thighs,huge calf,bad hands,fused hand,missing hand,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

参数设置：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

迭代步数 (Steps)：25文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

采样方法：DPM++ SDE Karras文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

重绘幅度：0.6文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

其他默认，点击生图。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

然后，用这张图作为底图，做一些变化，先把底图放到reference模式里面，操作步骤和参数同上，文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

然后修改正向提示词：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

功能1测试：更换衣服文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

正向词：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

photo of woman,wispy blonde hair,perfect eyes,a sexy swimsuit,(detailed face:1.2),(flat chest:1.1),slight smile,walking,outdoor,nature,beautiful,wild flowers,natural skin,dusk,orange clouds,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

8k uhd,high quality,ray tracing,(best quality),((masterpiece)),(highres),文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

功能2测试：更换背景，这边直接用泳装的正向提示词文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

正向词：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

photo of woman,wispy blonde hair,perfect eyes,a sexy swimsuit,(detailed face:1.2),(flat chest:1.1),slight smile,outdoor,in the swimming pool,beautiful,a bright sky,baiyun,natural skin,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

8k uhd,high quality,ray tracing,(best quality),((masterpiece)),(highres),文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

可以发现，它保持了底图的服装面料和主体的状态，效果堪比Lora。如果要更多更精确的图片可以批量生成，做挑选会更棒！文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

功能3测试：更换动作，这里要用到openpose控制模型搭配文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

正向词：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

photo of woman,wispy blonde hair,perfect eyes,simple long knitted tight-fitting shift dress,(detailed face:1.2),(flat chest:1.1),slight smile,outdoor,nature,beautiful,wild flowers,natural skin,dusk,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

8k uhd,high quality,ray tracing,(best quality),((masterpiece)),(highres),文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

场景的细节还是有点瑕疵，不过大体上还是与原底图匹配，如果觉得差别有点大，我们可以通过调整prompt提示词进行修改。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

另外还可以调整controlnet中，两个单元的引导介入时机和退出时机，比如这里，AI重绘基本是先根据openpose骨架图，确定任务的姿态，然后再通过稳定扩散分布进行生成，那么可以把reference介入时机可以调后一点，效果可能会更吻合。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

功能4测试：改变画风，文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

基于上面的图，做个2.5次元的看看吧，放到reference，换一个模型：Anything_V5，其他参数不变，测试看看。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

风格来说，还是比较稳定的，概率也比较大，可以通过调整提示词，生成更大量的图片，来进行筛选。想生成2次元的同学也是可以自己做一下测试，大家一起交流。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

5、现在测试一下二次元帅哥，效果如何。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

先在网上找一张图片，
文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

然后，用这张图作为底图，做一些变化，先把底图放到reference模式里面，操作步骤和参数同上，文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

换模型为Anything_V5等，调整提示词，文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

正向词：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

a boy,wearing clothes,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

(best quality),((masterpiece)),(highres),(an extremely delicate and beautiful),original,extremely detailed wallpaper,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

反向词：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

(naked:1.3),(multi-handed:1.4),worstquality,low quality,normal quality,jpegartifacts,signature,watermark,username,blurry,bad feet,bad_anatomy,bad_feet,bad_hands,bad_perspective,bad_proportions,error,extra_arms,extra_ears,extra_eyes,extra_mouth,feet_out_of_frame,foot_out_of_frame,greyscale,(head_out_of_frame:1.3),out-of-frame_censoring,out_of_frame,wrong_feet,文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

其他参数默认即可，点击生图文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

这里的关键词只写了：a boy,wearing clothes,一个穿着衣服的男孩，如果需要做更多精细化的调整，可以通过调整提示词来匹配，这样会更接近理想的效果。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

6、总结：文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html

reference对图片的扩散分布引导的效果还是比较不错的，如果是人像，搭配上roop插件，在保持画风等效果一致上面，其实还是很不错的，有炼丹Lora的同学，也可以将二者进行结合，相信可以更精准的控制图片的重绘导向。在电商的产品的应用上，也可以做一些尝试，保持产品统一，产品的场景图的风格的统一等等，上面也只是做一些基础的尝试，一个功能的革新，有时候是可以升级一次设计思维。文章源自菜鸟学院-https://www.cainiaoxueyuan.com/design/52342.html