^jbbdZddlZddlmZddlZddlZddlmZddlm Z ddl m Z ddl mZdd lmZmZmZdd lmZdd lmZmZdd lmZdd lmZddlmZmZmZm Z m!Z!ddl"m#Z#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,ddl-m.Z.ddl/m0Z0m1Z1ddl2m3Z3e,jhe5Z6dejnde8de8fdZ9GddejtZ; dGd%d&ejxZ?Gd'd(eZ@Gd)d*eZAe)Gd+d,e$ZBGd-d.eBZCGd/d0eBZDe)Gd1d2eBZEe)d34Gd5d6eBeZFGd7d8eBZGGd9d:eBeZHgd;ZIy)=z=PyTorch MarianMTModel model, ported from the Marian C++ repo.N)Callable)nn)CrossEntropyLoss)initialization)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)create_bidirectional_maskcreate_causal_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleis_torchdynamo_compilinglogging)merge_with_config_defaults)OutputRecordercapture_outputs) MarianConfig input_ids pad_token_iddecoder_start_token_idc|j|j}|ddddfj|ddddf<||dddf<| td|j |dk(||S)z1 Shift input ids one token to the right. Nr!rz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_)r#r$r%shifted_input_idss u/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/marian/modeling_marian.pyshift_tokens_rightr/8s}"++IOO<(CRC0668ae4adLMM""#4# )D92D> 9D> input_ids_shapepast_key_values_length position_idsc|F|dd\}}tj|||ztj|jj}t ||S)z3`input_ids_shape` is expected to be [bsz x seqlen].Nr>)r?device)rFarangelongrArWr9forward)r;rSrTrUbszseq_lenr<s r.rZz+MarianSinusoidalPositionalEmbedding.forward]s]  *2A.LC <<&(>(HPUPZPZcgcncncucuLw|,,r0N)rN) __name__ __module__ __qualname____doc__intr:rRrFno_gradSizeTensorrZ __classcell__r<s@r.r2r2HsNEcE#ECRVJEbfE U]]_pt -$zz -CF -Z_ZfZfimZm -  - -r0r2modulequerykeyvalueattention_maskscalingdropoutkwargsc ||jddz}tj||jdd|z}|||z}tj j |d}tj j|||j}tj||} | jddj} | |fS)Nr'r>rrLptrainingr!) sizerFmatmul transposer functionalsoftmaxrnru contiguous) rhrirjrkrlrmrnro attn_weights attn_outputs r.eager_attention_forwardr~ks**R.D(<<s}}Q':;gEL!#n4 ==((2(>L==((6??([L,,|U3K''1-88:K  $$r0ceZdZdZ ddedededededed edzd edzffd Z dd e jd e jdzde dzde jdzde e dee je jdzff dZxZS)MarianAttentionz=Multi-headed attention from 'Attention Is All You Need' paperN embed_dim num_headsrn is_decoderbias is_causalconfig layer_idxc t |||_||_||_||z|_||_|j |z|jk7rtd|jd|d|j dz|_||_ ||_ ||_ |9|jr-tjd|jjdt!j"||||_t!j"||||_t!j"||||_t!j"||||_y)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).rqzInstantiating a decoder z without passing `layer_idx` is not recommended and will lead to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.r)r9r:rrrnhead_dimrr+rmrrrlogger warning_oncer<r^rLineark_projv_projq_projout_proj) r;rrrnrrrrrr<s r.r:zMarianAttention.__init__s$ "" !Y.  MMI %$.. 8MdnnM]$YKr3 }}d* $""     *4>>+B+B*CD,,  ii 94@ ii 94@ ii 94@  )YTB r0 hidden_stateskey_value_statespast_key_valuesrlror6c |du}|jdd}g|d|j}|j|j|j dd} d} |St |t rA|jj|j} |r |j} n|j} n|} |r|n|} |rK|I| rG j|jj} | j|jj}n|j| } |j!| }g| jddd|j}| j|j dd} |j|j dd}|K j#| ||j\} }|r)t |t rd|j|j<t%j&|j(j*t,}||| | ||f|j.sdn |j0|j2d|\}}|j4g|dj7}|j9|}||fS) z#Input shape: Batch x Time x ChannelNr'r!r>FT)rnrm)r)rrviewrx isinstancer is_updatedgetrcross_attention_cacheself_attention_cachelayerskeysvaluesrrupdater get_interfacer_attn_implementationr~rurnrmreshaper{r)r;rrrrlrois_cross_attention input_shape hidden_shape query_statesrcurr_past_key_valuescurrent_states key_states value_stateskv_shapeattention_interfacer}r|s r.rZzMarianAttention.forwardsd.T9$))#2. 88b8$--8 {{=166|DNNqRST   &/+>?,77;;DNNK %+:+P+P(+:+O+O('6$-?)] /"=*-44T^^DIIJ/66t~~FMML^4J;;~6LF--cr2FBF FH#2<(?(M(M KK , ,.E) %8      % $}}C$,,LL %  % ! \*k));;;;FFH mmK0 L((r0)rFTFNNNNN)r^r_r`rarbfloatboolr"r:rFrer rrtuplerZrfrgs@r.rrsG  &* $%C%C%C %C  %C  %C%Ct#%C:%CT15(,.2 H)||H) ,,-H) H)  t+ H)-.H) u||U\\D00 1H)r0rceZdZd dededzffd Zdejdejdee dejfd Z xZ S) MarianEncoderLayerNrrcjt||j|_t |j|j |j |||_tj|j|_ |j|_ t|j|_|j|_tj |j|j"|_tj |j"|j|_tj|j|_y)N)rrrnrr)r9r:d_modelrrencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrnractivation_function activation_fnactivation_dropoutrencoder_ffn_dimfc1fc2final_layer_normr;rrr<s r.r:zMarianEncoderLayer.__init__s (nn44,,  %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r0rrlror6c F|}|j|fd|i|\}}tjj||j|j}||z}|j |}|}|j |j|}tjj||j|j}|j|}tjj||j|j}||z}|j|}|jtjk(rhtj|jsEtj |jj"dz }tj$|| |}|S)Nrlrsi)minmax)rrryrnrurrrrrrr?rFfloat16isfiniteallfinforclamp)r;rrlroresidual_ clamp_values r.rZzMarianEncoderLayer.forwardsh !)4>>  )   q  --mt||VZVcVc-d  =0 11-@  **488M+BC  --mt?V?Vaeanan-o /  --mt||VZVcVc-d  =0 --m<   %-- /}8U8Y8Y8[++m&9&9:>>EK!KK K<[YMr0r]) r^r_r`r"rbr:rFrHrrrerZrfrgs@r.rrs[=|=d =&(())+,   r0rceZdZddededzffd Z ddejdejdzdejdzdejdzd edzd e dzd e e d ejfd Z xZ S)MarianDecoderLayerNrrc t||j|_t |j|j |j dd|||_|j|_t|j|_ |j|_ tj|j|_t |j|j |j d|||_tj|j|_tj$|j|j&|_tj$|j&|j|_tj|j|_y)NT)rrrnrrrr)rnrrr)r9r:rrrdecoder_attention_headsrrrnrrrrrrr encoder_attnencoder_attn_layer_normrdecoder_ffn_dimrrrrs r.r:zMarianDecoderLayer.__init__3s  (nn44,, ~~ #F$>$>?"(";";$&LL$@!+ NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r0rrlencoder_hidden_statesencoder_attention_maskr use_cacheror6c |}|j|f||d|\}} tjj||j|j}||z}|j |}|h|}|j |f|||d|\}} tjj||j|j}||z}|j|}|}|j|j|}tjj||j|j}|j|}tjj||j|j}||z}|j|}|S)N)rrlrs)rrlr) rrryrnrurrrrrrrr) r;rrlrrrrrorrs r.rZzMarianDecoderLayer.forwardRs!*4>>  +)    q  --mt||VZVcVc-d  =0 11-@  ! ,$H0t00 !65 /      M1MM11-4<r r>rW)rlr#decoder_input_ids)rr$rFtensorrWne)r; pad_tokenr# dummy_inputss r.rz"MarianPreTrainedModel.dummy_inputssZKK,, LL"2Q2q)4L!MVZVaVab 'll95"!* r0)r^r_r`r"__annotations__base_model_prefixsupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraphrFrcrpropertyrrfrgs@r.rrsZ &*#N!U]]_22r0rceZdZdZeedZdeffd Ze e e d de jdzde jdzde jdzd eed ef d ZxZS) MarianEncoderz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`MarianEncoderLayer`]. Args: config: MarianConfig embed_tokens (nn.Embedding): output embedding )r attentionsrct|||j|_|j|_|j }|j |_|j|_ |jrtj|nd|_ tj|j ||j|_t%|j||j|_tj(t+|j,Dcgc] }t/|c}|_d|_|j5ycc}w)N?F)r9r:rnencoder_layerdrop layerdroprr$r5max_position_embeddingsmax_source_positionsscale_embeddingmathsqrt embed_scaler Embedding vocab_size embed_tokensr2embed_positions ModuleListrDencoder_layersrrgradient_checkpointing post_init)r;rrrr<s r.r:zMarianEncoder.__init__s  ~~ 11NN !..$*$B$B!393I3I499Y/sLL):):ItGWGWXB  * *It7G7G mmvOdOdIe$fA%7%?$fg &+#  %gsD;Nr#rl inputs_embedsror6c |du|duz r td||j||jz}|j|jdd}||z}t j j||j|j}t|j||}t|jD]F\}}d} |jr&tjg} | |jkrd} | r=|||fi|}Ht!|S)Nz:You must specify exactly one of input_ids or inputs_embedsr'rs)rr rlFT)last_hidden_state)r+rrrr)rryrnrur r enumeraterrFrandrr) r;r#rlr ro embed_posridx encoder_layerto_dropdropout_probabilitys r.rZzMarianEncoder.forwards   -t"; <YZ Z   --i84;K;KKM(()<)8N8N499V^^4TWLL)B)BFNNTXTdTdeB  * *FNND?  0 0   );;')+  ";;;'1"7 " ++  $&<<, & 4  --mt||VZVcVc-d "+DKK"8  C}}&+jjn#&7)%(> /# M  9++  r0)NNNNNNN)r^r_r`rarrrrr"r:rr rrFrrerHr rrrrrZrfrgs@r.rrs,$_A+V*?!P^_ |& .2.2:>:>(,26!%R ##d*R  t+R  %0047 R !& 0 04 7 R  R ((4/R $;R +,R  3R  R r0rceZdZddgZdeffd ZdZdZdZdZ d e d e jfd Z ee dd ej"d zdej$d zdej"d zdej$d zdeej$ezd zded zdej,d zdej,d zded zdeed efdZxZS) MarianModel$model.encoder.embed_positions.weight$model.decoder.embed_positions.weightrcPt|||j|j}}|jj r1t j||j||_ ddd|_ nd|_ t||_ t||_|jy)Nz shared.weight)zdecoder.embed_tokens.weightzencoder.embed_tokens.weight)r9r:r$rr share_encoder_decoder_embeddingsrrrshared_tied_weights_keysrencoderrdecoderr )r;rr5rr<s r.r:zMarianModel.__init__ws  "("5"5v7H7HZ  ;; 7 7,,z6>>;ODK/>/>'D # '+D #$V, $V,  r0c>|jjSr]) get_encoderget_input_embeddingsr;s r.r9z MarianModel.get_input_embeddingss!6688r0c|jjr>||_|j|j_|j|j _y||j_yr])rr2r3r5rr6r;rks r.set_input_embeddingsz MarianModel.set_input_embeddingssB ;; 7 7DK(, DLL %(, DLL %(-DLL %r0c|jjr td|jj S)Nz`get_decoder_input_embeddings` should not be called if `config.share_encoder_decoder_embeddings` is `True`. Please use `get_input_embeddings` instead.)rr2r+ get_decoderr9r:s r.get_decoder_input_embeddingsz(MarianModel.get_decoder_input_embeddingss< ;; 7 7H !6688r0ch|jjr td||j_y)Na `config.share_encoder_decoder_embeddings` is set to `True` meaning the decoder input embeddings are shared with the encoder. In order to set the decoder input embeddings, you should simply set the encoder input embeddings by calling `set_input_embeddings` with the appropriate embeddings.)rr2r+r6rr<s r.set_decoder_input_embeddingsz(MarianModel.set_decoder_input_embeddingss0 ;; 7 7r  %* !r0new_num_tokensr6c|jjr td|j}|j ||}|j ||j}||S||j_|j|SNz`resize_decoder_token_embeddings` should not be called if `config.share_encoder_decoder_embeddings` is `True`. Please use `resize_token_embeddings` instead.)rr2r+r@_get_resized_embeddingsrBr  tie_weights)r;rCold_embeddingsnew_embeddings model_embedss r.resize_decoder_token_embeddingsz+MarianModel.resize_decoder_token_embeddingss ;; 7 7K  ::<55nnU )).988:  ! *8 & r0Nr#rlrdecoder_attention_maskencoder_outputsrr decoder_inputs_embedsrroc ||jd |||d| }nGt|ts7t|dt|dkDr|dndt|dkDr|dnd}|jd |||d|||| d| } t | j | j| j| j| j|j |j|jS) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Marian uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. Example: ```python >>> from transformers import AutoTokenizer, MarianModel >>> tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de") >>> model = MarianModel.from_pretrained("Helsinki-NLP/opus-mt-en-de") >>> inputs = tokenizer("Studies have been shown that owning a dog is good for you", return_tensors="pt") >>> decoder_inputs = tokenizer( ... " Studien haben gezeigt dass es hilfreich ist einen Hund zu besitzen", ... return_tensors="pt", ... add_special_tokens=False, ... ) >>> outputs = model(input_ids=inputs.input_ids, decoder_input_ids=decoder_inputs.input_ids) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 26, 512] ```N)r#rlr rr!r>)rrrr#rlrrrr r)rrdecoder_hidden_statesdecoder_attentionsrencoder_last_hidden_staterencoder_attentions) r5rrlenr6rrrrrr) r;r#rlrrLrMrr rNrrodecoder_outputss r.rZzMarianModel.forwardsh  "*dll#-+ OO_=-"1!"4474H14Loa0RV14_1E1I?1-tO'$,,  '1"1!"4#1+/    "-??+;;"1"?"?.99,==&5&G&G"1"?"?.99  r0) NNNNNNNNN)r^r_r`_keys_to_ignore_on_load_missingr"r:r9r=r@rBrbrrrKrrrFrrerrr rHrrrrrZrfrgs@r.r.r.ps[ /.'# |*9.9*cbll0.2.2596:HL(,26:>!%V ##d*V  t+V !++d2 V !& t 3 V u||,>E V V ((4/V  %0047V $;V +,V  V V r0r.zX The Marian Model with a language modeling head. Can be used for summarization. ) custom_introcFeZdZdZgdZddgZddiZdeffd Z d!d e d e d zd e d e jffd Z d"d e d e jfdZdZd e d d fdZde jfdZee d#dej*d zdej,d zdej*d zdej,d zdeej,ezd zded zdej4d zdej4d zdej*d zde d zdeed efdZdej,fd ZxZ S)$rr)rr/r0r/r0lm_head.weight!model.decoder.embed_tokens.weightrct||t||_|jj r dddd|_|j r |jn |j}|jdtjd|ftj|j|d|_|j!y)Nzmodel.shared.weight)r[r\z!model.encoder.embed_tokens.weightrr!Fr)r9r:r.rrr2r4rr register_bufferrFzerosrrrlm_headr )r;rtarget_vocab_sizer<s r.r:zMarianMTModel.__init__+s   ( ;; 7 7"75J5J'D # 281X1XF--^d^w^w 0%++qBS>T2UVyy1BO  r0NrCpad_to_multiple_of mean_resizingr6cxt||||}|jjr|j ||Sr])r9resize_token_embeddingsrr2_resize_final_logits_bias)r;rCrbrcrIr<s r.rez%MarianMTModel.resize_token_embeddings<s;8I[]jk ;; 7 7  * *> :r0c|j}|j|||}|j||jjd}|j j r||j _|j j rY|jI|j js3|j}|j||}|j||jS)Nr) r9rFr=rAr)rr2r get_output_embeddingstie_word_embeddings_get_resized_lm_headset_output_embeddings)r;rCrbargsrHrI old_lm_head new_lm_heads r._resize_token_embeddingsz&MarianMTModel._resize_token_embeddingsEs22455nnVhi !!.1'..44Q7 ;; 7 7-;DKK * KK 8 8**,8KK33446K33KPK  & &{ 3((**r0c(|jjr td|jj }|j ||}|jj ||jI|jjs3|j}|j||}|j||jj }||S||j_ |j|j||SrE)rr2r+rr@rFrBrhrirjrkr rGrf)r;rCrHrIrmrnrJs r.rKz-MarianMTModel.resize_decoder_token_embeddings[s ;; 7 7K  @@B55nnU //?  % % ' 3DKK<[<[446K33KPK  & &{ 3zz>>@  ! *8 &  &&~6r0c6|jjd}||kr|jddd|f}nStjd||z f|jj}tj |j|gd}|j d|y)Nr'r!rrrr)rr)rFr_rWcatr^)r;rCold_num_tokensnew_bias extra_biass r.rfz'MarianMTModel._resize_final_logits_bias{s//55b9 ^ +--a..@AHa.)H%IRVRhRhRoRopJyy$"8"8*!E1MH 0(;r0rIc||_yr])r`)r;rIs r.rkz#MarianMTModel.set_output_embeddingss % r0r#rlrrLrMrr rNlabelsrroc l| R| rtjdd} |7|5t| |jj|jj }|j |f|||||||| d| } |j| d|jz} d}| Ft}|| jd|jj| jd}t|| | j| j| j| j | j"| j$| j& S)u decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Marian uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MarianMTModel >>> src = "fr" # source language >>> trg = "en" # target language >>> model_name = f"Helsinki-NLP/opus-mt-{src}-{trg}" >>> model = MarianMTModel.from_pretrained(model_name) >>> tokenizer = AutoTokenizer.from_pretrained(model_name) >>> sample_text = "où est l'arrêt de bus ?" >>> batch = tokenizer([sample_text], return_tensors="pt") >>> generated_ids = model.generate(**batch) >>> tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] "Where's the bus stop?" ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F)rlrrMrLrr rNrrr') losslogitsrrQrRrrSrrT)rwarningr/rr$r%rr`rrrr rrrQrRrrSrrT)r;r#rlrrLrMrr rNrwrrooutputs lm_logitsmasked_lm_lossloss_fcts r.rZzMarianMTModel.forwards@p  klI (-B-J$6DKK44dkk6X6X%!'1djj  ' )/+#9+'"7 '  ' LL,t/E/EE   ')H%innR9W9W&XZ`ZeZefhZijN#33")"?"?&99$55&-&G&G")"?"?&99  r0clt||jj|jjSr])r/rr$r%)r;rws r.%prepare_decoder_input_ids_from_labelsz3MarianMTModel.prepare_decoder_input_ids_from_labelss%!&$++*B*BDKKDfDfggr0)NTr]) NNNNNNNNNN)!r^r_r`rrX_keys_to_ignore_on_saver4r"r:rbrrrrerorKrfrkrrrFrrerrr rHrrrrZrrfrgs@r.rrs  '# FGmn*,OP|$ae!7:TzY] +s+_a_k_k+,@<<<&BLL&.2.2596:HL(,26:>*.!%] ##d*]  t+] !++d2 ] !& t 3 ] u||,>E ] ] ((4/]  %0047]   4'] $;] +,]  ] ] ~hELLhr0rc(eZdZdZfdZdZxZS)MarianDecoderWrapperz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cdt||t||_|j yr])r9r:rr6r r;rr<s r.r:zMarianDecoderWrapper.__init__s&  $V,  r0c&|j|i|Sr])r6)r;rlros r.rZzMarianDecoderWrapper.forwardst||T,V,,r0)r^r_r`rar:rZrfrgs@r.rrs  -r0rc\eZdZddiZfdZdZdZee dde jdzde jdzd e jdzd e jdzd e dzd e jdzd e jdzdedzdee jzdeedeezfdZxZS)MarianForCausalLMr[r\cd|_d|_t| |t ||_t j|j|jd|_ |jy)NTFr) rr$r9r:rrrr hidden_sizerr`r rs r.r:zMarianForCausalLM.__init__sX $)!  )&1 yy!3!3V5F5FUS  r0cB|jjjSr]rr6rr:s r.r9z&MarianForCausalLM.get_input_embeddings szz!!...r0c:||jj_yr]rr<s r.r=z&MarianForCausalLM.set_input_embeddingss*/ 'r0Nr#rlrrrr rwrlogits_to_keepror6c |jjd|||||||d| } | d} t| tr t | dn| } |j | dd| ddf}d}|a|j |j}t}||jd|jj|jd}t||| j| j| j| j S)aT labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MarianForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-fr-en") >>> model = MarianForCausalLM.from_pretrained("Helsinki-NLP/opus-mt-fr-en") >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```rPrNr')ryrzrrrrrU)rr6rrbslicer`torWrrrrrrrrr)r;r#rlrrrr rwrrror|r slice_indicesrzryrs r.rZzMarianForCausalLM.forwardsL>PTZZ=O=O > )"7#9+' >  >   8B>SV8W~ot4]k mA}a,?@A  YYv}}-F')HFKKDKK,B,BCV[[QS_UD0#33!//))$55   r0) NNNNNNNNr)r^r_r`r4r:r9r=rrrFrrerHr rrbrrrrrZrfrgs@r.rrs/= /0.2.2:>;?(,26*.!%-.A ##d*A  t+A  %0047 A !& 1 1D 8 A  A ((4/A   4'A $;A ell*A +,A  2 2A A r0r)rr.rr)Nr)Jrarcollections.abcrnumpyrBrFrtorch.nnrrr activationsr cache_utilsr r r generationr masking_utilsr rmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrr utils.genericrutils.output_capturingrr configuration_marianr" get_loggerr^rrerbr/rr2Modulerr~rrrrrrr.rrr__all__rUr0r.rsD $ %&!CC)JB9G&8E.   H % %,, c [^  -",,-R!% II% <<% % << % LL4' % T\ %%'(%:r)biir)l030hO3OdO