^jddlZddlmZmZejeZerddlZ d dejdejdejdejde d e d ejfd Z d d Z y)N)is_torchaudio_availablelogginglogitstargets logit_lengthstarget_lengthsblank_token_id reductionreturnc ts tdd}||vr(td|ddjd|Dd|j |j }t jj|jj|j |j j|j |j j|j|d }|d k(r/|j|jjz S|d k(r|jS|d k(r!||jz jS|d k(r|jS|S)a Compute standard RNN-T (RNN Transducer) loss (https://huggingface.co/papers/1211.3711). Thin wrapper around [`torchaudio.functional.rnnt_loss`]. torchaudio is queried with `reduction="none"` to get the per-sample negative log-likelihoods, and the requested reduction is applied here. The reduction names and formulas mirror NeMo's `RNNTLoss` (the reference implementation used to train/finetune Parakeet), so that loss magnitudes and gradient scaling match when finetuning other RNNT models like Parakeet: - `"mean_volume"`: sum of per-sample losses divided by the sum of target lengths (per-token average over the whole batch). This is what `nvidia/parakeet-rnnt-0.6b` is trained with (`rnnt_reduction: mean_volume`). - `"mean_batch"`: plain average of per-sample losses over the batch (NeMo's default). - `"mean"`: per-sample loss divided by its own target length, then averaged over the batch. - `"sum"`: sum of per-sample losses. - `"none"`: per-sample losses, unreduced. Args: logits: Joint token logits of shape `(batch, T, U+1, vocab_size)`. targets: Target labels of shape `(batch, U)`. logit_lengths: Encoder output lengths of shape `(batch,)`. target_lengths: Target lengths of shape `(batch,)`. blank_token_id: Blank token id. reduction: Loss reduction method. One of `"mean_volume"`, `"mean_batch"`, `"mean"`, `"sum"`, or `"none"`. Returns: Scalar loss tensor (or per-example losses if `reduction="none"`). zWComputing the RNN-T loss requires torchaudio. Install it with `pip install torchaudio`.) mean_volume mean_batchmeansumnonezInvalid reduction mode "z". Expected one of z, c32K|]}t|yw)N)repr).0rs f/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/loss/loss_rnnt.py zrnnt_loss..DsNq[\tTUwNqs.r)rrrr blankr rrrr)r ImportError ValueErrorjointodevice torchaudio functional rnnt_lossfloat contiguousintrr)rrrr r r valid_reductionslossess rr"r"s`H # $sttK((&yk1DTYYNq`pNqEqDrrs t  $&&v}}5N  " " , ,||~((* 6==)--/#&&v}}599;%))+ -FM!zz|n22488::: l "{{} f --//5577 e zz| Mc $t||||||S)N)rrrr r r )r")rlabelsr label_lengthsr r kwargss rParakeetForRNNTLossr-\s# #$%  r()r) torchutilsrr get_logger__name__loggerr Tensorr%strr"r-r(rr6s 4   H %# ? LL? \\?<<?LL ?  ?  ? \\?P r(