/;jiddlZddlmcmZddlmZddlmZddlmZerddl Z ej ej ej ej ej dZdedejfd ZGd d ejZGd d ejZ GddejZGddejZdS)N)nn) deprecate)is_torch_npu_available)swishsilumishgelureluact_fnreturncz|}|tvr t|Std|)zHelper function to get activation function from string. Args: act_fn (str): Name of activation function. Returns: nn.Module: Activation function. z!Unsupported activation function: )lowerACTIVATION_FUNCTIONS ValueError)r s V/root/voice-cloning/.venv/lib/python3.11/site-packages/diffusers/models/activations.pyget_activationr$s?\\^^F %%%#F++EVEEFFFcFeZdZdZfdZdejdejfdZxZS)FP32SiLUzH SiLU activation function with input upcasted to torch.float32. cHtdSN)super__init__)self __class__s rrzFP32SiLU.__init__:s rinputsr ctj|d|jS)NF)inplace)Frfloattodtype)rrs rforwardzFP32SiLU.forward=s/vfllnne44477 EEEr) __name__ __module__ __qualname____doc__rtorchTensorr$ __classcell__rs@rrr5srFelFu|FFFFFFFFrrc `eZdZdZd dedededeffd Zd ej d ej fd Z d Z xZ S)GELUa GELU activation function with tanh approximation support with `approximate="tanh"`. Parameters: dim_in (`int`): The number of channels in the input. dim_out (`int`): The number of channels in the output. approximate (`str`, *optional*, defaults to `"none"`): If `"tanh"`, use tanh approximation. bias (`bool`, defaults to True): Whether to use a bias in the linear layer. noneTdim_indim_out approximatebiascttj||||_||_dSNr3)rrrLinearprojr2)rr0r1r2r3rs rrz GELU.__init__Ls? IfgD999 &rgater c|jjdkrtj||jStj|t j|j|jS)Nmps)r2r#) devicetyper r r2r"r)float32r#rr9s rr z GELU.geluQsg ; u $ $6$D,<=== =vdggEMg22@PQQQTT[_[eTfffrcZ||}||}|Sr)r8r )r hidden_statess rr$z GELU.forwardWs* -00  -00 r)r/T) r%r&r'r(intstrboolrr)r*r r$r+r,s@rr.r.As''s'S's'SW'''''' gg%,gggg rr.c\eZdZdZd dededeffd Zdejdejfd Z d Z xZ S) GEGLUaF A [variant](https://arxiv.org/abs/2002.05202) of the gated linear unit activation function. Parameters: dim_in (`int`): The number of channels in the input. dim_out (`int`): The number of channels in the output. bias (`bool`, defaults to True): Whether to use a bias in the linear layer. Tr0r1r3cttj||dz||_dS)Nrr6rrrr7r8rr0r1r3rs rrzGEGLU.__init__gs9 Ifgk=== rr9r c|jjdkrtj|Stj|t j|jS)Nr;r<)r=r>r r r"r)r?r#r@s rr z GEGLU.geluksS ; u $ $6$<< vdggEMg223366TZ6HHHrc^t|dks|ddd}tdd|||}t rt j|dddS|dd \}}|||zS) NrscalezThe `scale` argument is deprecated and will be ignored. Please remove it, as passing it will raise an error in the future. `scale` should directly be passed while calling the underlying pipeline component i.e., via `cross_attention_kwargs`.z1.0.0)dimr2r)rP) lengetrr8r torch_npu npu_gegluchunkr )rrBargskwargsdeprecation_messager9s rr$z GEGLU.forwardqs t99q==FJJw55A#U  gw(; < < < -00 ! # # 3&}"!LLLQO O"/"5"5aR"5"@"@ M4 499T??2 2rT) r%r&r'r(rCrErr)r*r r$r+r,s@rrGrG]s>>s>S>>>>>>>II%,IIII 3 3 3 3 3 3 3rrGcVeZdZdZd dededeffd Zdejdejfd Z xZ S) ApproximateGELUa The approximate form of the Gaussian Error Linear Unit (GELU). For more details, see section 2 of this [paper](https://arxiv.org/abs/1606.08415). Parameters: dim_in (`int`): The number of channels in the input. dim_out (`int`): The number of channels in the output. bias (`bool`, defaults to True): Whether to use a bias in the linear layer. Tr0r1r3cttj||||_dSr5rIrJs rrzApproximateGELU.__init__s5 IfgD999 rxr c`||}|tjd|zzS)NgZd;?)r8r)sigmoid)rr]s rr$zApproximateGELU.forwards* IIaLL5=++++rrY) r%r&r'r(rCrErr)r*r$r+r,s@rr[r[~s::s:S:::::::,,%,,,,,,,,,rr[)r)torch.nn.functionalr functionalr utilsrutils.import_utilsrrSSiLUMishr.ReLUrrDModulerrrGr[rrris  777777RWYY BGII BGII BGII BGII G3G29GGGG" F F F F Fry F F F29833333BI333B,,,,,bi,,,,,r