PhPddlmZddlZddlmZddlmZddlm Z m Z m Z m Z m Z mZddlZddlmZmZmZmZmZmZmZmZ m!Z"m#Z$ddl%m&Z'm(Z)m*Z+ddl,m-Z- dd l.m/Z0gd Z2ed gd Z3d Z4dZ5Gdde Z6GddeZ7GddeZ8GddZ9dddddddde:ddddddf d"dZ;d#d Z_get_word_dictrLAsI( 9$%g.  *tBx3MRS^SiSiSkMl mMl[ " "1 %)>)>q)A(B'C CMl m zl( %a(  ns*Bc@eZdZUdZded< ded< ded< ded <y ) rzORepresents hypothesis generated by CTC beam search decoder :class:`CTCDecoder`.torch.LongTensorr$z List[str]wordsfloatr,torch.IntTensor timestepsN)__name__ __module__ __qualname____doc____annotations__r>rrPs,Z h  L+nrYrcBeZdZdZedfd Zdfd ZddZxZS)rzLanguage model state.ct|S)zMap of indices to LM states)superchildren)self __class__s r>r]zCTCDecoderLMState.childrenhswrYc"t||S)a!Returns child corresponding to usr_index, or creates and returns a new state if input index is not found. Args: usr_index (int): index corresponding to child state Returns: CTCDecoderLMState: child state corresponding to usr_index )r\child)r^ usr_indexr_s r>razCTCDecoderLMState.childmsw}Y''rYcy)zCompare two language model states. Args: state (CTCDecoderLMState): LM state to compare against Returns: int: 0 if the states are the same, -1 if self is less, +1 if self is greater. NrXr^states r>comparezCTCDecoderLMState.compareys rY)returnzDict[int, CTCDecoderLMState])rbintrgr)rerrgr) rSrTrUrVpropertyr]rarf __classcell__)r_s@r>rres"    (  rYrcFeZdZdZeddZeddZeddZy) rzVLanguage model base class for creating custom language models to use with the decoder.ct)zInitialize or reset the language model. Args: start_with_nothing (bool): whether or not to start sentence with sil token. Returns: CTCDecoderLMState: starting state NotImplementedError)r^start_with_nothings r>r)zCTCDecoderLM.starts "!rYct)axEvaluate the language model based on the current LM state and new word. Args: state (CTCDecoderLMState): current LM state usr_token_idx (int): index of the word Returns: (CTCDecoderLMState, float) CTCDecoderLMState: new LM state float: score rm)r^re usr_token_idxs r>r,zCTCDecoderLM.scores "!rYct)a8Evaluate end for language model based on current LM state. Args: state (CTCDecoderLMState): current LM state Returns: (CTCDecoderLMState, float) CTCDecoderLMState: new LM state float: score rmrds r>finishzCTCDecoderLM.finishs "!rYN)roboolrgr)rerrqrhrgTuple[CTCDecoderLMState, float])rerrgru)rSrTrUrVrr)r,rsrXrYr>rrs?` " """  " "rYrceZdZdZ d dZddZddZdZdZddZ ddZ dd Z d dd Z dd Z y )rzCTC beam search decoder from *Flashlight* :cite:`kahn2022flashlight`. .. devices:: CPU Note: To build the decoder, please use the factory function :func:`ctc_decoder`. c ||_||_||_|jj||_|jj|} g} |rKt ||||| } |j| } d} t || || |j| | | |_||_ yt||| |j| |_||_ y)a Args: nbest (int): number of best decodings to return lexicon (Dict or None): lexicon mapping of words to spellings, or None for lexicon-free decoder word_dict (_Dictionary): dictionary of words tokens_dict (_Dictionary): dictionary of tokens lm (CTCDecoderLM): language model. If using a lexicon, only word level LMs are currently supported decoder_options (_LexiconDecoderOptions or _LexiconFreeDecoderOptions): parameters used for beam search decoding blank_token (str): token corresopnding to blank sil_token (str): token corresponding to silence unk_word (str): word corresponding to unknown FN) nbestr2r1r+blankr?_LexiconDecoderdecoder_LexiconFreeDecoderr%)r^rxr#r2r1r%decoder_options blank_token sil_tokenrIr3 transitionsr5token_lms r>__init__zCTCDecoder.__init__s4 "&%%// < "",,Y7 "; 7BPD **84HH*  DL" /GTZZYdeDL rYcdtj|D}tfd|}tjt |S)Nc3&K|] }|d yw)rNrX).0gs r> z)CTCDecoder._get_tokens..s/.!.sc"|jk7SrA)ry)xr^s r>z(CTCDecoder._get_tokens..s TZZrY)itgroupbyfiltertorch LongTensorlist)r^idxss` r> _get_tokenszCTCDecoder._get_tokenss8/bjj.//6T ++rYcg}t|D]7\}}||jk(r|dk(s |||dz k7s'|j|9tj|S)z8Returns frame numbers corresponding to non-blank tokens.r) enumerateryappendr IntTensor)r^rrRrJidxs r>_get_timestepszCTCDecoder._get_timestepss_ oFAsdjj AvQU +  # & y))rYc8|jjy)aInitialize the internal state of the decoder. See :py:meth:`decode_step` for the usage. .. note:: This method is required only when performing online decoding. It is not necessary when performing batch decoding with :py:meth:`__call__`. N)r{ decode_beginr^s r>rzCTCDecoder.decode_begins !!#rYc8|jjy)aFinalize the internal state of the decoder. See :py:meth:`decode_step` for the usage. .. note:: This method is required only when performing online decoding. It is not necessary when performing batch decoding with :py:meth:`__call__`. N)r{ decode_endrs r>rzCTCDecoder.decode_ends !rYc|jtjk7r td|js t d|j s t d|jdk7rt d|j|j\}}|jj|j||y)aPerform incremental decoding on top of the curent internal state. .. note:: This method is required only when performing online decoding. It is not necessary when performing batch decoding with :py:meth:`__call__`. Args: emissions (torch.FloatTensor): CPU tensor of shape `(frame, num_tokens)` storing sequences of probability distribution over labels; output of acoustic model. Example: >>> decoder = torchaudio.models.decoder.ctc_decoder(...) >>> decoder.decode_begin() >>> decoder.decode_step(emission1) >>> decoder.decode_step(emission2) >>> decoder.decode_end() >>> result = decoder.get_final_hypothesis() emissions must be float32.emissions must be a CPU tensor.emissions must be contiguous.zemissions must be 2D. Found N) dtyperfloat32 ValueErroris_cpu RuntimeError is_contiguousndimshapesizer{ decode_stepdata_ptr)r^ emissionsTNs r>rzCTCDecoder.decode_step s( ??emm +9: :@A A&&(>? ? >>Q !=ioo=NOP P~~1   !3!3!5q!_to_hypozCTCDecoder._to_hypoCs" " '' 6get_final_hypothesiszCTCDecoder.get_final_hypothesisNs0,,779}}W\tzz233rYNc|jtjk7r td|js t d|j s t d|jdk7rt d|j||js t d|j\}}}|tj|f|}d}g}t|D]w}|j||z|jd zz} |jj| |||} |j!|j#| d|j$y|S) a Performs batched offline decoding. .. note:: This method performs offline decoding in one go. To perform incremental decoding, please refer to :py:meth:`decode_step`. Args: emissions (torch.FloatTensor): CPU tensor of shape `(batch, frame, num_tokens)` storing sequences of probability distribution over labels; output of acoustic model. lengths (Tensor or None, optional): CPU tensor of shape `(batch, )` storing the valid length of in time axis of the output Tensor in each batch. Returns: List[List[CTCHypothesis]]: List of sorted best hypotheses for each audio sequence in the batch. rrrzemissions must be 3D. Found Nzlengths must be a CPU tensor.r)rrrrrrrrrrfullrFrstrider{decoderrrx) r^rlengthsBrr float_byteshyposb emissions_ptrrs r>__call__zCTCDecoder.__call__]s5, ??emm +9: :@A A&&(>? ? >>Q !=ioo=NOP P  w~~>? ?.."1a ?jj!q)G qA%..0;?YEUEUVWEX3XXMll))-QGG LLw|'<= > rYcz|Dcgc]+}|jj|j-c}Scc}w)z Map raw token IDs into corresponding tokens Args: idxs (LongTensor): raw token IDs generated from decoder Returns: List: tokens corresponding to the input IDs )r1rGitem)r^rrs r>idxs_to_tokenszCTCDecoder.idxs_to_tokenss5CGG$3  **388:6$GGGs08)rxrhr#zOptional[Dict]r2rBr1rBr%rr}z9Union[_LexiconDecoderOptions, _LexiconFreeDecoderOptions]r~rErrErIrErgNone)rrQrgrN)rrQrgrQ)rtorch.FloatTensor)rgzList[CTCHypothesis]rA)rrrzOptional[torch.Tensor]rgzList[List[CTCHypothesis]])rrNrgr)rSrTrUrVrrrrrrrrrrrXrYr>rrs77 7 7 ! 7  7S7777 7r, * $ "!=F   4 OS0*05K0 "0d HrYrr2rz-infF-|zc |t|tur tdt|}|r@t |}t ||xs|j ||| | | | tj }n2t||xs|j ||| | tj}t|||||}t|tk(rt tdt||}n | t}t||||||| || S)aY Builds an instance of :class:`CTCDecoder`. Args: lexicon (str or None): lexicon file containing the possible words and corresponding spellings. Each line consists of a word and its space separated spelling. If `None`, uses lexicon-free decoding. tokens (str or List[str]): file or list containing valid tokens. If using a file, the expected format is for tokens mapping to the same index to be on the same line lm (str, CTCDecoderLM, or None, optional): either a path containing KenLM language model, custom language model of type `CTCDecoderLM`, or `None` if not using a language model lm_dict (str or None, optional): file consisting of the dictionary used for the LM, with a word per line sorted by LM index. If decoding with a lexicon, entries in lm_dict must also occur in the lexicon file. If `None`, dictionary for LM is constructed using the lexicon file. (Default: None) nbest (int, optional): number of best decodings to return (Default: 1) beam_size (int, optional): max number of hypos to hold after each decode step (Default: 50) beam_size_token (int, optional): max number of tokens to consider at each decode step. If `None`, it is set to the total number of tokens (Default: None) beam_threshold (float, optional): threshold for pruning hypothesis (Default: 50) lm_weight (float, optional): weight of language model (Default: 2) word_score (float, optional): word insertion score (Default: 0) unk_score (float, optional): unknown word insertion score (Default: -inf) sil_score (float, optional): silence insertion score (Default: 0) log_add (bool, optional): whether or not to use logadd when merging hypotheses (Default: False) blank_token (str, optional): token corresponding to blank (Default: "-") sil_token (str, optional): token corresponding to silence (Default: "|") unk_word (str, optional): word corresponding to unknown (Default: "") Returns: CTCDecoder: decoder Example >>> decoder = ctc_decoder( >>> lexicon="lexicon.txt", >>> tokens="tokens.txt", >>> lm="kenlm.bin", >>> ) >>> results = decoder(emissions) # List of shape (B, nbest) of Hypotheses z!lm_dict must be None or str type.) beam_sizebeam_size_tokenbeam_threshold lm_weight word_score unk_score sil_scorelog_addcriterion_type)rrrrrrrzflashlight-text is installed, but KenLM is not installed. Please refer to https://github.com/kpu/kenlm#python-module for how to install it.) rxr#r2r1r%r}r~rrI)rDrErrB _load_words_LexiconDecoderOptionsr'_CriterionTypeCTC_LexiconFreeDecoderOptionsrL_KenLMr_ZeroLMr)r#r$r%rHrxrrrrrrrrr~rrIr1r}r2s r>rrs"rtG}C7<==f%Kg&0+G{/E/E/G)!)--  5+G{/E/E/G))-- wG[(KI Bx3 >d B "  Y  '  rYcv|dvrt|dd|}t|d|d|dk7r |dSdS) N) librispeechzlibrispeech-3-gramzlibrispeech-4-gramzZ not supported. Must be one of ['librispeech-3-gram', 'librispeech-4-gram', 'librispeech']zdecoder-assets/z /lexicon.txtz /tokens.txtrz/lm.binr")r_PretrainedFiles)modelprefixs r>_get_filenamesr ss OOgo p  ug &F (,'%!&-!7fXW  >B rYct|}t|j}t|j}|jt|j}nd}t |||S)aM Retrieves pretrained data files used for :func:`ctc_decoder`. Args: model (str): pretrained language model to download. Valid values are: ``"librispeech-3-gram"``, ``"librispeech-4-gram"`` and ``"librispeech"``. Returns: Object with the following attributes * ``lm``: path corresponding to downloaded language model, or ``None`` if the model is not associated with an lm * ``lexicon``: path corresponding to downloaded lexicon file * ``tokens``: path corresponding to downloaded tokens file Nr")rrr#r$r%r)rfiles lexicon_file tokens_filelm_files r>r r s]" 5 !E!%--0L .K xx *   rY)"r# Optional[str]r$zUnion[str, List[str]]r%zUnion[str, CTCDecoderLM]rHrrxrhrrhrz Optional[int]rrPrrPrrPrrPrrPrrtr~rErrErIrErgr)rrErgr)> __future__r itertoolsrabcr collectionsrtypingrrrr r r rflashlight.lib.text.decoderr rr rzrrrr|rrr_LMr_LMStaterr/rr(rrflashlight.lib.text.dictionaryrrCrrBrrtorchaudio.utilsr!flashlight.lib.text.decoder.kenlmrr Exception__all__rr?rLrrrrrPrrr rXrYr>rs""AA     ,A /1LM  oJo*  @."3."bcHcHR$(!%)V}!n n !n !n n  n  n#nnnnnnnnn !n"#nb w? s6CC:#C*)C:*C41C:3C44C:9C: