1;ji 5 dZddlZddlmZeeZGddejZddd d d d d ddddd Zddddddddddddd Z ddddddddddd Z idddd dd!dd"dd#dd$dd%dd&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6Z dd d!d"d#d$d%d&d7d8d9d:d; Z dZ eje eje iZeje eje iZeje iZd?d@iZdAZdGdBZdGdCZdDZdEZdGdFZdS)HzI State dict utilities: utility methods for converting state dicts easily N) get_loggerc"eZdZdZdZdZdZdZdS) StateDictTypez6 The mode to use when converting state dicts. diffusers_oldkohya_sspeft diffusersN)__name__ __module__ __qualname____doc__ DIFFUSERS_OLDKOHYA_SSPEFT DIFFUSERSZ/root/voice-cloning/.venv/lib/python3.11/site-packages/diffusers/utils/state_dict_utils.pyrrs.$MH DIIIrrz.to_out.0.lora_Bz.to_out.0.lora_Az .to_q.lora_Az .to_q.lora_Bz .to_k.lora_Az .to_k.lora_Bz .to_v.lora_Az .to_v.lora_Bz.lora_B.lora_Az.to_out.0.lora_magnitude_vector) .to_out_lora.up.to_out_lora.down.to_q_lora.down .to_q_lora.up.to_k_lora.down .to_k_lora.up.to_v_lora.down .to_v_lora.upz.lora.upz .lora.down.to_out.lora_magnitude_vectorz.q_proj.lora_Bz.q_proj.lora_Az.k_proj.lora_Bz.k_proj.lora_Az.v_proj.lora_Bz.v_proj.lora_Az.out_proj.lora_Bz.out_proj.lora_Aztext_projection.lora_A.weightztext_projection.lora_B.weight) .q_proj.lora_linear_layer.up.q_proj.lora_linear_layer.down.k_proj.lora_linear_layer.up.k_proj.lora_linear_layer.down.v_proj.lora_linear_layer.up.v_proj.lora_linear_layer.down.out_proj.lora_linear_layer.up .out_proj.lora_linear_layer.down.lora_linear_layer.up.lora_linear_layer.downz text_projection.lora.down.weightztext_projection.lora.up.weight) rrrrrrrrr(r)r r!r"r#r$r%r&r'z to_k.lora_Azto_k.lora.downz to_k.lora_Bz to_k.lora.upz to_q.lora_Azto_q.lora.downz to_q.lora_Bz to_q.lora.upz to_v.lora_Azto_v.lora.downz to_v.lora_Bz to_v.lora.upzto_out.0.lora_Azto_out.0.lora.downzto_out.0.lora_Bzto_out.0.lora.upz.k_proj.lora_magnitude_vectorz.v_proj.lora_magnitude_vectorz.q_proj.lora_magnitude_vectorz.out_proj.lora_magnitude_vector) rrrrrrrrz.to_k.lora_magnitude_vectorz.to_v.lora_magnitude_vectorz.to_q.lora_magnitude_vectorr lora_downlora_up)lora_Alora_Bz .processor..cDi}|D]\}}tD])}||vr#t|}|||}*|D]&}||vr ||}|||}n'|||<|S)a& Simply iterates over the state dict and replaces the patterns in `mapping` with the corresponding values. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. mapping (`dict[str, str]`): The mapping to use for conversion, the mapping should be a dictionary with the following structure: - key: the pattern to replace - value: the pattern to replace with Returns: converted_state_dict (`dict`) The converted state dict. )itemsKEYS_TO_ALWAYS_REPLACEkeysreplace) state_dictmappingconverted_state_dictkvpattern new_patterns rconvert_state_dictr;s   "" $ $1-2244 4 4G!||4W= IIg{33||~~  G!||%g. IIg{33#$Q rc |td|Dr tj}nGtd|Dr tj}nt d|t vrt d|dt |}t||S)a Converts a state dict to the PEFT format The state dict can be from previous diffusers format (`OLD_DIFFUSERS`), or new diffusers format (`DIFFUSERS`). The method only supports the conversion from diffusers old/new to PEFT for now. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. original_type (`StateDictType`, *optional*): The original type of the state dict, if not provided, the method will try to infer it automatically. Nc3K|]}d|vV dS to_out_loraNr.0r7s r z-convert_state_dict_to_peft..'==a}!======rc3K|]}d|vV dSlora_linear_layerNrr@s rrBz-convert_state_dict_to_peft..(EEa$)EEEEEEr-Could not automatically infer state dict typeOriginal type  is not supported)anyr2rrr ValueErrorPEFT_STATE_DICT_MAPPINGSr;)r4 original_typekwargsr5s rconvert_state_dict_to_peftrPs ==:??+<+<=== = = N)7MM EE:??3D3DEEE E E N)3MMLMM M499;;;;J-JJJKKK&}5G j' 2 22rc :|dddznd|td|Dr tj}nvtfd|Dr tj}nrr@s rrBz2convert_state_dict_to_diffusers..rCrc3(K|] }dd|vV dSrz.weightNrrAr7peft_adapter_names rrBz2convert_state_dict_to_diffusers..s5VVq5,555:VVVVVVrc3K|]}d|vV dSrErr@s rrBz2convert_state_dict_to_diffusers..rGrrHrIrJ) poprKr2rrrrLDIFFUSERS_STATE_DICT_MAPPINGSr;)r4rNrOr5rXs @rconvert_state_dict_to_diffusersr\s3, >488$"33 ==:??+<+<=== = = N)7MM VVVVJOODUDUVVV V V N).MM EE:??3D3DEEE E E N LMM M9>>@@@@J-JJJKKK+M:G j' 2 22rc0t}t||S)za Converts a state dict from UNet format to diffusers format - i.e. by removing some keys )UNET_TO_DIFFUSERSr;)r4r5s rconvert_unet_state_dict_to_peftr_s G j' 2 22rc t|}n;#t$r.}t|dkrt|}nYd}~nd}~wwxYwt d|Dst d|S)z Attempts to first `convert_state_dict_to_peft`, and if it doesn't detect `lora_linear_layer` for a valid `DIFFUSERS` LoRA for example, attempts to exclusively convert the Unet `convert_unet_state_dict_to_peft` rHNc3&K|] }d|vpd|vV dS)r,r-Nr)rAkeys rrBz1convert_all_state_dict_to_peft.. s/NNcx31(c/NNNNNNrz#Your LoRA was not converted to PEFT)rP Exceptionstrr_rKr2rL)r4 peft_dictes rconvert_all_state_dict_to_peftrgs .z::  q66D D D7 CCII  IIII NNY^^=M=MNNN N N@>??? s A $AA c  ddl}n)#t$rtdwxYw|dd d z nd |9t fd|Dr tj}|tvrtd|d t|ttj}i}| D]\}}d |vr| d d }nPd |vr| d d }n5d|vr| dd}nd|vr| dd}| dd|ddz }| d}|||<d|vrC|ddd}|t#|||<|S)a Converts a `PEFT` state dict to `Kohya` format that can be used in AUTOMATIC1111, ComfyUI, SD.Next, InvokeAI, etc. The method only supports the conversion from PEFT to Kohya for now. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. original_type (`StateDictType`, *optional*): The original type of the state dict, if not provided, the method will try to infer it automatically. kwargs (`dict`, *args*): Additional arguments to pass to the method. - **adapter_name**: For example, in case of PEFT, some keys will be pre-pended with the adapter name, therefore needs a special handling. By default PEFT also takes care of that in `get_peft_model_state_dict` method: https://github.com/huggingface/peft/blob/ba0477f2985b1ba311b83459d29895c809404e99/src/peft/utils/save_and_load.py#L92 but we add it here in case we don't want to rely on that method. rNzDConverting PEFT state dicts to Kohya requires torch to be installed.rRr.rSc3(K|] }dd|vV dSrVrrWs rrBz.convert_state_dict_to_kohya..3s5TTQ3*333q8TTTTTTrrIrJztext_encoder_2.z lora_te2.z text_encoder.z lora_te1.unet lora_unetlora_magnitude_vector dora_scale_r*z.alpha)torch ImportErrorloggererrorrZrKr2rrKOHYA_STATE_DICT_MAPPINGSrLr;r0r3countsplittensorlen) r4rNrOrpkohya_ss_partial_state_dictkohya_ss_state_dict kohya_keyweight alpha_keyrXs @rconvert_state_dict_to_kohyar~sI&  [\\\  >488$"33 TTTT*//BSBSTTT T T /).M5::<<<<J-JJJKKK#5ZAZ[h[mAn"o"o9>>@@GG 6  ) )!))*;[IIII  ) )!))/;GGII y !))&+>>II $ 1 1!))*A<PPI%%c3 0D0Dq0HII %%&7<< )/I& ) # #$??3//2:::I-2\\#f++-F-F  * s&.)N)renumloggingrr rrEnumrr^DIFFUSERS_TO_PEFTDIFFUSERS_OLD_TO_PEFTPEFT_TO_DIFFUSERSDIFFUSERS_OLD_TO_DIFFUSERSPEFT_TO_KOHYA_SSrrrMrr[rtr1r;rPr\r_rgr~rrrrs  H  DI*+%#%#%#%F   %5&6$4&6$4&6&8(:&((G&E   &'%'%')+&(  4646  4  6 8:#>#>#>+ )!(4737377;#B#B#B%F   !6. !;)! +/1AB3    B33336,3,3,3,3^333&<<<<<<r