h94

/

IP-Adapter-FaceID

like 1.85k

I tried to load this into the AUTOMATIC1111 web ui via the ControlNet extension, but got some errors. Can this be used like other IP Adapter models or can it only be used in code (per the README.md)?

2023-12-31 08:07:03,631 - ControlNet - INFO - Loaded state_dict from [E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\models\ip-adapter-faceid-plusv2_sd15.bin]
*** Error running process: E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "E:\github\stable-diffusion-webui\modules\scripts.py", line 718, in process
        script.process(p, *script_args)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1053, in process
        self.controlnet_hack(p)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1042, in controlnet_hack
        self.controlnet_main_entry(p)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 758, in controlnet_main_entry
        model_net = Script.load_control_model(p, unet, unit.model)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 364, in load_control_model
        model_net = Script.build_control_model(p, unet, model)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 393, in build_control_model
        network = build_model_by_guess(state_dict, unet, model_path)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet_model_guess.py", line 244, in build_model_by_guess
        network = PlugableIPAdapter(state_dict)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlmodel_ipadapter.py", line 334, in __init__
        self.ipadapter = IPAdapterModel(state_dict,
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlmodel_ipadapter.py", line 212, in __init__
        self.load_ip_adapter(state_dict)
      File "E:\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlmodel_ipadapter.py", line 215, in load_ip_adapter
        self.image_proj_model.load_state_dict(state_dict["image_proj"])
      File "E:\github\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
        raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
    RuntimeError: Error(s) in loading state_dict for MLPProjModel:
        Missing key(s) in state_dict: "proj.3.weight", "proj.3.bias".
        Unexpected key(s) in state_dict: "norm.weight", "norm.bias", "perceiver_resampler.proj_in.weight", "perceiver_resampler.proj_in.bias", "perceiver_resampler.proj_out.weight", "perceiver_resampler.proj_out.bias", "perceiver_resampler.norm_out.weight", "perceiver_resampler.norm_out.bias", "perceiver_resampler.layers.0.0.norm1.weight", "perceiver_resampler.layers.0.0.norm1.bias", "perceiver_resampler.layers.0.0.norm2.weight", "perceiver_resampler.layers.0.0.norm2.bias", "perceiver_resampler.layers.0.0.to_q.weight", "perceiver_resampler.layers.0.0.to_kv.weight", "perceiver_resampler.layers.0.0.to_out.weight", "perceiver_resampler.layers.0.1.0.weight", "perceiver_resampler.layers.0.1.0.bias", "perceiver_resampler.layers.0.1.1.weight", "perceiver_resampler.layers.0.1.3.weight", "perceiver_resampler.layers.1.0.norm1.weight", "perceiver_resampler.layers.1.0.norm1.bias", "perceiver_resampler.layers.1.0.norm2.weight", "perceiver_resampler.layers.1.0.norm2.bias", "perceiver_resampler.layers.1.0.to_q.weight", "perceiver_resampler.layers.1.0.to_kv.weight", "perceiver_resampler.layers.1.0.to_out.weight", "perceiver_resampler.layers.1.1.0.weight", "perceiver_resampler.layers.1.1.0.bias", "perceiver_resampler.layers.1.1.1.weight", "perceiver_resampler.layers.1.1.3.weight", "perceiver_resampler.layers.2.0.norm1.weight", "perceiver_resampler.layers.2.0.norm1.bias", "perceiver_resampler.layers.2.0.norm2.weight", "perceiver_resampler.layers.2.0.norm2.bias", "perceiver_resampler.layers.2.0.to_q.weight", "perceiver_resampler.layers.2.0.to_kv.weight", "perceiver_resampler.layers.2.0.to_out.weight", "perceiver_resampler.layers.2.1.0.weight", "perceiver_resampler.layers.2.1.0.bias", "perceiver_resampler.layers.2.1.1.weight", "perceiver_resampler.layers.2.1.3.weight", "perceiver_resampler.layers.3.0.norm1.weight", "perceiver_resampler.layers.3.0.norm1.bias", "perceiver_resampler.layers.3.0.norm2.weight", "perceiver_resampler.layers.3.0.norm2.bias", "perceiver_resampler.layers.3.0.to_q.weight", "perceiver_resampler.layers.3.0.to_kv.weight", "perceiver_resampler.layers.3.0.to_out.weight", "perceiver_resampler.layers.3.1.0.weight", "perceiver_resampler.layers.3.1.0.bias", "perceiver_resampler.layers.3.1.1.weight", "perceiver_resampler.layers.3.1.3.weight".
        size mismatch for proj.0.weight: copying a param with shape torch.Size([1024, 512]) from checkpoint, the shape in current model is torch.Size([512, 512]).
        size mismatch for proj.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([512]).
        size mismatch for proj.2.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([768, 512]).
        size mismatch for proj.2.bias: copying a param with shape torch.Size([3072]) from checkpoint, the shape in current model is torch.Size([768]).