0;ji dZddlZddlmZmZmZddlmZGddejjZGddejZ dS) z*Swish() activation function for Conformer.N)nnsinpow) Parameterc6eZdZdZdejdejfdZdS)SwishzConstruct an Swish object.xreturnc0|tj|zS)z!Return Swish activation function.)torchsigmoid)selfr s h/root/voice-cloning/.venv/lib/python3.11/site-packages/chatterbox/models/s3gen/transformer/activation.pyforwardz Swish.forwards5=####N)__name__ __module__ __qualname____doc__r Tensorrrrrrs@$$$$%,$$$$$$rrc*eZdZdZdfd ZdZxZS)Snakea Implementation of a sine-based periodic activation function Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter References: - This activation function is from this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snake(256) >>> x = torch.randn(256) >>> x = a1(x) ?TFcLtt|||_||_|jr*t t j||z|_n)t t j ||z|_||j_ d|_ dS)a  Initialization. INPUT: - in_features: shape of the input - alpha: trainable parameter alpha is initialized to 1 by default, higher values = higher-frequency. alpha will be trained along with the rest of your model. g& .>N) superr__init__ in_featuresalpha_logscalerr zerosalphaones requires_gradno_div_by_zero)rrr!alpha_trainabler __class__s rrzSnake.__init__2s eT##%%%&-   D"5;{#;#;e#CDDDJJ"5:k#:#:U#BCCDJ#2  )rc|jdd}|jrtj|}|d||jzz t t||zdzz}|S)u Forward pass of the function. Applies the function to the input elementwise. Snake ∶= x + 1/a * sin^2 (xa) rr)r! unsqueezerr expr$rr)rr r!s rrz Snake.forwardIsv  $$Q''11"55   %Ie$$E  334CE NNA8N8NN Nr)rTF)rrrrrr __classcell__)r&s@rrr"sV******.       rr) rr rrrtorch.nnrModulerrrrrr/s"10 $$$$$EHO$$$22222BI22222r