0;ji#ddlmZmZddlZddlZddlZddlmcm Z ddl m Z ddl mZmZdZdZdZdZd ZGd d eZdS) )ListTupleN)padding) S3TokenizerV2 ModelConfigi>iic eZdZdZdZdefdedeffd Zdee j fdZ d Z e j dd e j d d dedee j e jffdZ dde j defdZxZS) S3Tokenizerz s3tokenizer.S3TokenizerV2 with the following changes: - a more integrated `forward` - compute `log_mel_spectrogram` using `_mel_filters` and `window` in `register_buffers` ) _mel_filterswindowspeech_tokenizer_v2_25hznameconfigcdt|d|_tjt |j|j}|dtj ||dtj |jdS)Ni)srn_fftn_melsr r ) super__init__rlibrosafiltersmelS3_SRrregister_buffertorch FloatTensor hann_window)selfrrr __class__s c/root/voice-cloning/.venv/lib/python3.11/site-packages/chatterbox/models/s3tokenizer/s3tokenizer.pyrzS3Tokenizer.__init__s  ***=+     l + +      dj ) )     returncg}|D]}t|tjrtj|}|dkr|d}|jd|z tz}tj |}||tz z}t|}tj j |d||jdz fdd}|||S)z Given a list of wavs with the same `sample_rate`, pad them so that the length is multiple of 40ms (S3 runs at 25 token/sec). rconstant)modevalue) isinstancenpndarrayr from_numpydim unsqueezeshape S3_TOKEN_RATEceilintnn functionalpadappend)rwavsrprocessed_wavswavn_tokensintended_wav_lens r!r6zS3Tokenizer.pad6s ' 'C#rz** ,&s++wwyyA~~mmA&& ! r)]:Hwx((H'2 +=> "#344 (%))$sy}45 *C  ! !# & & & &r"cg}|D]r}t|tjrtj|}|dkr|d}||s|S)z4Prepare a list of audios for s3tokenizer processing.r%r)r*r+r,rr-r.r/r7)rr8r9r:s r!_prepare_audiozS3Tokenizer._prepare_audioNsy ' 'C#rz** ,&s++wwyyA~~mmA&&  ! !# & & & &r"Nr8 accelerator Acceleratormax_lenc`||}gg}}|D]j}||j}||}||dd|dzf}||dkt |\}}||} n||} | |||j\} } | | fS)a NOTE: mel-spec has a hop size of 160 points (100 frame/sec). FIXME: this class inherits `nn.Module` but doesn't accept `torch.Tensor` and handles a list of wavs one by one, which is unexpected. Args ---- - `wavs`: 16 kHz speech audio - `max_len` max length to truncate the output sequence to (25 token/sec). NOTE: please pad the waveform if longer sequence is needed. N.r) r>todevicelog_mel_spectrogramr7squeezer unwrap_modelquantizelongdetach) rr8r?rAr9melsmel_lensr:r tokenizer speech_tokensspeech_token_lenss r!forwardzS3Tokenizer.forwardZs+",,T22Rh! ( (C&&%%C**3//C"#|! |+, KK A ' ' ' ' h  II#0066I+4+=+=dHKKPTP[D\D\+]+]( (    ' ' ) )  " " $ $ + + - -  r"raudiorc~tj|stj|}||j}|dkrt j|d|f}tj||jt|j |jd}|dddf dz}|j |j|z}tj |d }tj||d z }|d zd z }|S) a Compute the log-Mel spectrogram of Parameters ---------- audio: torch.Tensor, shape = (*) The path to audio or either a NumPy array or Tensor containing the audio waveform in 16 kHz padding: int Number of zero samples to pad to the right Returns ------- torch.Tensor, shape = (128, n_frames) A Tensor that contains the Mel spectrogram rT)r return_complex.Nr&g|=)ming @g@)r is_tensorr-rDrEFr6stftrS3_HOPr absr clamplog10maximummax)rrRrrY magnitudesmel_speclog_specs r!rFzS3Tokenizer.log_mel_spectrograms,u%% ,$U++E%% Q;;E%!W..Ez 4:v;>>$+..    #ss(^''))1, $'' 44zA;xU33399;;=8<<>>C+?@@sNc)r")NN)r)__name__ __module__ __qualname____doc__ignore_state_dict_missingrstrrrrTensorr6r>no_gradr3r LongTensorrQrF __classcell__)r s@r!r r sD !;-)kmm         .tEL10   U]__$( # # l# ##  # u|U-- . # # # _# P((|(((((((((r"r )typingrrnumpyr+rrtorch.nn.functionalr4r5rXs3tokenizer.utilsrs3tokenizer.model_v2rrrrZ S3_TOKEN_HOPr1SPEECH_VOCAB_SIZEr r"r!rus %%%%%%    RRRRR-RRRRRr"