^jldZddlmZddlZddlmZddlmZmZmZddl m Z ddl m Z dd lmZdd lmZmZmZmZmZmZdd lmZmZdd lmZdd lmZddlmZm Z m!Z!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(e"jRe*Z+GddejXZ-GddejXZ.GddejXZ/GddejXZ0GddejXZ1GddejXZ2Gdd ejXZ3Gd!d"ejXZ4Gd#d$ejXZ5 dCd%ejXd&ejld'ejld(ejld)ejldzd*e7d+e7fd,Z8Gd-d.ejXZ9Gd/d0ejXZ:Gd1d2eZ;Gd3d4ejXZe Gd9d:e=Z?e d;<Gd=d>e=Z@e d?<Gd@dAe=ZAgdBZBy)DzPyTorch MarkupLM model.)CallableN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingMaskedLMOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)apply_chunking_to_forward)TransformersKwargsauto_docstringcan_return_tuplelogging)merge_with_config_defaults)capture_outputs)MarkupLMConfigc*eZdZdZfdZddZxZS)XPathEmbeddingszConstruct the embeddings from xpath tags and subscripts. We drop tree-id in this version, as its info can be covered by xpath. c t||j|_tj|j |jz|j |_tj|j|_ tj|_ tj|j |jzd|j z|_ tjd|j z|j |_tjt!|jDcgc],}tj"|j$|j .c}|_tjt!|jDcgc],}tj"|j(|j .c}|_ycc}wcc}w)N)super__init__ max_depthrLinearxpath_unit_hidden_size hidden_sizexpath_unitseq2_embeddingsDropouthidden_dropout_probdropoutReLU activationxpath_unitseq2_inner inner2emb ModuleListrange Embeddingmax_xpath_tag_unit_embeddingsxpath_tag_sub_embeddingsmax_xpath_subs_unit_embeddingsxpath_subs_sub_embeddings)selfconfig_ __class__s y/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/markuplm/modeling_markuplm.pyr#zXPathEmbeddings.__init__3sZ )))+63P3PSWSaSa3acicucu)v&zz&"<"<= '')$&IIf.K.Kdnn.\^_bhbtbt^t$u!1v'9'9#96;M;MN(* t~~.  VAA6C`C`a ) %*,t~~.  VBBFDaDab * &   s 11G1Gc g}g}t|jD]^}|j|j||dddd|f|j|j||dddd|f`t j |d}t j |d}||z}|j|j|j|j|}|S)Ndim) r1r$appendr4r6torchcatr/r+r-r.)r7xpath_tags_seqxpath_subs_seqxpath_tags_embeddingsxpath_subs_embeddingsixpath_embeddingss r;forwardzXPathEmbeddings.forwardMs " "t~~& eA ! ( ()I)F)Fq)I.YZ\]_`Y`Ja)b c ! ( ()J)G)G)J>Z[]^`aZaKb)c d e!& *?R H % *?R H03HH>>$,,ttG`G`aqGr7s*tu)NN)__name__ __module__ __qualname____doc__r#rI __classcell__r:s@r;rr-s  4 rJrcXeZdZdZfdZedZeddZ ddZxZ S)MarkupLMEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.cdt|||_tj|j |j |j|_tj|j|j |_ |j|_ t||_ tj|j|j |_tj |j |j"|_tj$|j&|_|j+dt-j.|jj1dd|j|_tj|j|j |j2|_ y)N) padding_idxeps position_idsrr=F) persistent)r"r#r8rr2 vocab_sizer' pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsr$rrHtype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsr)r*r+register_bufferrAarangeexpandrTr7r8r:s r;r#zMarkupLMEmbeddings.__init__bs:  !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c )) / 7%'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<=   ELL)G)GHOOPWXej  "..#%<<  * *F,>,>DL\L\$  rJc|jdd}|d}tj|dz||zdztj|j}|j dj |S)z We are provided embeddings directly. We cannot infer which are padded so just generate sequential position ids. Args: inputs_embeds: torch.Tensor Returns: torch.Tensor Nr=rdtypedevicer)sizerArdlongrj unsqueezere) inputs_embedsrT input_shapesequence_lengthrWs r;&create_position_ids_from_inputs_embedsz9MarkupLMEmbeddings.create_position_ids_from_inputs_embedszsp$((*3B/ %a.|| !O_{:Q>ejjYfYmYm %%a(// <)neintrAcumsumtype_asrl) input_idsrTpast_key_values_lengthmaskincremental_indicess r;"create_position_ids_from_input_idsz5MarkupLMEmbeddings.create_position_ids_from_input_idssW||K(,,.$||Da8@@FI__cgg"'')K77rJc||j}n|jdd}| |jn |j}|;||j||j}n|j ||j}|&t j |t j|}||j|}|]|jjt jtt||jgzt j|z}|]|jjt jtt||jgzt j|z}|} |j!|} |j#|} |j%||} | | z| z| z} |j'| } |j)| } | S)Nr=rh)rkrjr{rTrqrAzerosrlr\r8 tag_pad_idonestuplelistr$ subs_pad_idr^r`rHrar+)r7rwrCrDtoken_type_idsrWrnrorjwords_embeddingsr^r`rH embeddingss r;rIzMarkupLMEmbeddings.forwards  #..*K',,.s3K%.%:!! @T@T  $#FFyRVRbRbc #JJ=Z^ZjZjk  !"[[EJJvVN   00;M  !![[33ejjd;'4>>*::;5::V\7N  !![[44uzzd;'4>>*::;5::V\8N)"66|D $ : :> J00P%(;;>SSVff ^^J/ \\*- rJ)r)NNNNNN) rKrLrMrNr# staticmethodrqr{rIrOrPs@r;rRrR_sLQ 0==" 8 8"1rJrRcneZdZfdZdej dej dej fdZxZS)MarkupLMSelfOutputc(t|tj|j|j|_tj |j|j|_tj|j|_ yNrU) r"r#rr%r'denserarbr)r*r+rfs r;r#zMarkupLMSelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= rJ hidden_states input_tensorreturncr|j|}|j|}|j||z}|SNrr+rar7rrs r;rIzMarkupLMSelfOutput.forward7 =1  ]3 }|'CD rJrKrLrMr#rATensorrIrOrPs@r;rr1> U\\RWR^R^rJrcVeZdZfdZdej dej fdZxZS)MarkupLMIntermediatect|tj|j|j |_t|jtrt|j|_ y|j|_ yr) r"r#rr%r'intermediate_sizer isinstance hidden_actstrr intermediate_act_fnrfs r;r#zMarkupLMIntermediate.__init__s] YYv1163K3KL f'' -'-f.?.?'@D $'-'8'8D $rJrrcJ|j|}|j|}|Sr)rrr7rs r;rIzMarkupLMIntermediate.forwards& =1 00? rJrrPs@r;rrs#9U\\ellrJrcneZdZfdZdej dej dej fdZxZS)MarkupLMOutputc(t|tj|j|j |_tj|j |j|_tj|j|_ yr) r"r#rr%rr'rrarbr)r*r+rfs r;r#zMarkupLMOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= rJrrrcr|j|}|j|}|j||z}|Srrrs r;rIzMarkupLMOutput.forwardrrJrrPs@r;rrrrJrcVeZdZfdZdej dej fdZxZS)MarkupLMPoolerct|tj|j|j|_tj |_yr)r"r#rr%r'rTanhr-rfs r;r#zMarkupLMPooler.__init__s9 YYv1163E3EF '')rJrrc\|dddf}|j|}|j|}|S)Nr)rr-)r7rfirst_token_tensor pooled_outputs r;rIzMarkupLMPooler.forwards6+1a40 #56  6 rJrrPs@r;rrs#$ U\\ellrJrcVeZdZfdZdej dej fdZxZS)MarkupLMPredictionHeadTransformcht|tj|j|j|_t |jtrt|j|_ n|j|_ tj|j|j|_ yr) r"r#rr%r'rrrrr transform_act_fnrarbrfs r;r#z(MarkupLMPredictionHeadTransform.__init__s{ YYv1163E3EF f'' -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STrJrrcl|j|}|j|}|j|}|Sr)rrrars r;rIz'MarkupLMPredictionHeadTransform.forwards4 =1 --m< }5 rJrrPs@r;rrs$UU\\ellrJrc$eZdZfdZdZxZS)MarkupLMLMPredictionHeadct|t||_t j |j |jd|_t jtj|j|_ y)NT)bias) r"r#r transformrr%r'rZdecoder ParameterrAr}rrfs r;r#z!MarkupLMLMPredictionHead.__init__$s[ 8@yy!3!3V5F5FTR LLV->->!?@ rJcJ|j|}|j|}|Sr)rrrs r;rIz MarkupLMLMPredictionHead.forward-s$}5  ]3 rJ)rKrLrMr#rIrOrPs@r;rr#sArJrcVeZdZfdZdej dej fdZxZS)MarkupLMOnlyMLMHeadcBt|t||_yr)r"r#r predictionsrfs r;r#zMarkupLMOnlyMLMHead.__init__5s 3F;rJsequence_outputrc(|j|}|Sr)r)r7rprediction_scoress r;rIzMarkupLMOnlyMLMHead.forward9s ,,_=  rJrrPs@r;rr4s#<!u||! !rJrmodulequerykeyvalueattention_maskscalingr+c tj||jdd|z}|||z}tjj |dtj j|j}tjj|||j}tj||} | jddj} | |fS)Nrr=)r?ri)ptrainingr) rAmatmul transposer functionalsoftmaxfloat32torir+r contiguous) rrrrrrr+kwargs attn_weights attn_outputs r;eager_attention_forwardr?s<<s}}Q':;gEL!#n4 ==((2U]](SVVW\WbWbcL==((6??([L,,|U3K''1-88:K  $$rJc eZdZfdZ ddej dej dzdeede ej ej dzffdZ xZ S) MarkupLMSelfAttentionc$t||j|jzdk7r2t |ds&t d|jd|jd||_|j|_t|j|jz |_|j|jz|_ tj|j|j|_ tj|j|j|_ tj|j|j|_tj|j |_|j |_|jdz|_y)Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()g)r"r#r'num_attention_headshasattr ValueErrorr8rtattention_head_size all_head_sizerr%rrrr)attention_probs_dropout_probr+attention_dropoutrrfs r;r#zMarkupLMSelfAttention.__init__WsC     : : :a ?PVXhHi#F$6$6#78 445Q8   #)#=#= #&v'9'9F $))#2. CCbC$*B*BC zz-055lCMMaQRS XXm,11,?II!QO zz-055lCMMaQRS (?(M(M KK , ,.E) %8      % $}}C$2H2HLL %  % ! \*k));;;;FFH L((rJr) rKrLrMr#rAr FloatTensorrrrrIrOrPs@r;rrVsd6048)||)))D0)+, ) u||U\\D00 1 )rJrc eZdZfdZ ddej dej dzdeedej fdZ xZ S) MarkupLMAttentioncbt|t||_t ||_yr)r"r#rr7routputrfs r;r#zMarkupLMAttention.__init__s& )&1 (0 rJNrrrrc ^|}|j|fd|i|\}}|j||}|SNr)r7r)r7rrrresidualr9s r;rIzMarkupLMAttention.forwardsK !$499  )   q  M8< rJr) rKrLrMr#rArrrrrIrOrPs@r;rrsQ148 || ))D0 +,   rJrc eZdZfdZ d dej dej dzdeedej fdZ dZ xZ S) MarkupLMLayerct||j|_d|_t ||_t ||_t||_ y)Nr) r"r#chunk_size_feed_forward seq_len_dimr attentionr intermediaterrrfs r;r#zMarkupLMLayer.__init__sI '-'E'E$*6208$V, rJNrrrrc |j|fd|i|}t|j|j|j|}|Sr)rrfeed_forward_chunkrr)r7rrrs r;rIzMarkupLMLayer.forwardsY '  )  2  # #T%A%A4CSCSUb rJcL|j|}|j||}|Sr)rr)r7attention_outputintermediate_output layer_outputs r;rz MarkupLMLayer.feed_forward_chunks,"//0@A{{#68HI rJr) rKrLrMr#rArrrrrIrrOrPs@r;rrsV-48||))D0+,   $rJrc neZdZfdZ ddej dej dzdeede fdZ xZ S) MarkupLMEncoderct|||_tjt |j Dcgc] }t|c}|_d|_ ycc}w)NF) r"r#r8rr0r1num_hidden_layersrlayergradient_checkpointing)r7r8rGr:s r;r#zMarkupLMEncoder.__init__sN  ]]5IaIaCb#caM&$9#cd &+#$dsA#Nrrrrc P|jD] }|||fi|}t|S)N)last_hidden_state)rr )r7rrr layer_modules r;rIzMarkupLMEncoder.forwardsC !JJ L(M +  rJr) rKrLrMr#rArrrrr rIrOrPs@r;rrsM,48 || ))D0 +,    rJrc`eZdZUeed<dZeedZe jfdZ xZ S)MarkupLMPreTrainedModelr8markuplm)r attentionscXt||t|tr t j |j yt|trZt j|jtj|jjdjdyy)zInitialize the weightsr=rXN)r" _init_weightsrrinitzeros_rrRcopy_rWrArdrre)r7rr:s r;rz%MarkupLMPreTrainedModel._init_weightssx f% f6 7 KK $  2 3 JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h4rJ) rKrLrMr__annotations__base_model_prefixrr_can_record_outputsrAno_gradrrOrPs@r;r r s; "&+ U]]_iirJr cHeZdZdfd ZdZdZeee dde jdzde jdzde jdzde jdzd e jdzd e jdzd e jdzd e e d eezfdZxZS) MarkupLMModelct||||_t||_t ||_|r t|nd|_|jy)zv add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) r"r#r8rRrrencoderrpooler post_init)r7r8add_pooling_layerr:s r;r#zMarkupLMModel.__init__sM   ,V4&v. 0AnV,t  rJc.|jjSrrr\)r7s r;get_input_embeddingsz"MarkupLMModel.get_input_embeddingss...rJc&||j_yrr )r7rs r;set_input_embeddingsz"MarkupLMModel.set_input_embeddingss*/'rJNrwrCrDrrrWrnrrc | | td|#|j|||j} n!||jdd} n td| |jn |j} |t j | | }|&t j | tj| }|jdjd} | j|j } d | z d z} |j|||||| } |j| | fi|} | d }|j|j|nd}t||S)a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. Examples: ```python >>> from transformers import AutoProcessor, MarkupLMModel >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> model = MarkupLMModel.from_pretrained("microsoft/markuplm-base") >>> html_string = " Page Title " >>> encoding = processor(html_string, return_tensors="pt") >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 4, 768] ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer=z5You have to specify either input_ids or inputs_embeds)rjrhrr)rig?g)rwrCrDrWrrnr)r  pooler_output)r%warn_if_padding_and_no_attention_maskrkrjrArr}rlrmrrirrrr )r7rwrCrDrrrWrnrrorjextended_attention_maskembedding_outputencoder_outputsrrs r;rIzMarkupLMModel.forward suJ  ]%>cd d  "  6 6y. Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ FCN  !"[[EJJvVN"0":":1"="G"G"J"9"<"<4::"<"N#&)@#@H"L??))%)' + '$,,  #   *!,8< 8O O4UY )-'  rJ)T)NNNNNNN)rKrLrMr#r!r#rrrrA LongTensorrrrrr rIrOrPs@r;rrs  /0 .2262637260426K ##d*K ((4/K ((4/ K ))D0 K ((4/ K &&-K ((4/K +,K  + +K  K rJrceZdZfdZee ddejdzdejdzdejdzdejdzdejdzdejdzd ejdzd ejdzd ejdzd ee d e eje zfdZ xZ S)MarkupLMForQuestionAnsweringct|||j|_t|d|_t j |j|j|_|jyNF)r) r"r# num_labelsrr rr%r' qa_outputsrrfs r;r#z%MarkupLMForQuestionAnswering.__init___sU   ++%fF ))F$6$68I8IJ rJNrwrCrDrrrWrnstart_positions end_positionsrrc |j|f||||||d| } | d} |j| } | jdd\}}|jdj }|jdj }d}|| t |j dkDr|jd}t | j dkDr| jd} |j d}|jd|| jd|t|}|||}||| }||zdz }t|||| j| j S) ae xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. Examples: ```python >>> from transformers import AutoProcessor, MarkupLMForQuestionAnswering >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base-finetuned-websrc") >>> model = MarkupLMForQuestionAnswering.from_pretrained("microsoft/markuplm-base-finetuned-websrc") >>> html_string = " My name is Niels " >>> question = "What's his name?" >>> encoding = processor(html_string, questions=question, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> predict_answer_tokens = encoding.input_ids[0, answer_start_index : answer_end_index + 1] >>> processor.decode(predict_answer_tokens).strip() 'Niels' ```rCrDrrrWrnrrr=r>N) ignore_indexr)loss start_logits end_logitsrr) r r0splitsqueezerlenrkclamp_rrrr)r7rwrCrDrrrWrnr1r2routputsrlogitsr7r8 total_loss ignored_indexloss_fct start_lossend_losss r;rIz$MarkupLMForQuestionAnswering.forwardisZ $--   ))))%'    "!*1#)<<r<#: j#++B/::< ''+668   &=+D?'')*Q."1"9"9""==%%'(1, - 5 5b 9 (--a0M  " "1m 4  M 2']CH!,@J M:H$x/14J+%!!//))   rJ) NNNNNNNNN)rKrLrMr#rrrArrrrrrIrOrPs@r;r,r,\s *..2.2.2.2,0-1/3-1T <<$&T  t+T  t+ T  t+ T  t+ T llT)T ||d*T ,T ||d*T +,T  u|| ; ;T T rJr,zC MarkupLM Model with a `token_classification` head on top. ) custom_introcjeZdZfdZee ddejdzdejdzdejdzdejdzdejdzdejdzd ejdzd ejdzd ee d e eje zfd Z xZ S)MarkupLMForTokenClassificationcdt|||j|_t|d|_|j |j n |j }tj||_ tj|j|j|_ |jyr.)r"r#r/rr classifier_dropoutr*rr)r+r%r' classifierrr7r8rHr:s r;r#z'MarkupLMForTokenClassification.__init__s   ++%fF )/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ rJNrwrCrDrrrWrnlabelsrrc <|j|f||||||d| } | d} |j| } d} |Ft}|| jd|jj |jd} t | | | j| jS)a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. Examples: ```python >>> from transformers import AutoProcessor, AutoModelForTokenClassification >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> processor.parse_html = False >>> model = AutoModelForTokenClassification.from_pretrained("microsoft/markuplm-base", num_labels=7) >>> nodes = ["hello", "world"] >>> xpaths = ["/html/body/div/li[1]/div/span", "/html/body/div/li[1]/div/span"] >>> node_labels = [1, 2] >>> encoding = processor(nodes=nodes, xpaths=xpaths, node_labels=node_labels, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> loss = outputs.loss >>> logits = outputs.logits ```r4rNr=r6r>rr) r rIrrr8r/rrr)r7rwrCrDrrrWrnrKrr=rrr6rAs r;rIz&MarkupLMForTokenClassification.forwardsV $--   ))))%'    "!* OOO<  ')H!&&r4;;+A+AB BD %$!//))   rJNNNNNNNN)rKrLrMr#rrrArrrrrrIrOrPs@r;rFrFs *..2.2.2.2,0-1&*D <<$&D  t+D  t+ D  t+ D  t+ D llT)D ||d*D  t#D +,D  u|| ~ -D D rJrFz MarkupLM Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. cjeZdZfdZee ddejdzdejdzdejdzdejdzdejdzdejdzd ejdzd ejdzd ee d e eje zfd Z xZ S)!MarkupLMForSequenceClassificationcnt|||j|_||_t ||_|j |j n |j}tj||_ tj|j|j|_ |jyr)r"r#r/r8rr rHr*rr)r+r%r'rIrrJs r;r#z*MarkupLMForSequenceClassification.__init__(s   ++ %f- )/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ rJNrwrCrDrrrWrnrKrrc |j|f||||||d| } | d} |j| } |j| } d} ||jj|j dk(rd|j_nl|j dkDrL|j tjk(s|j tjk(rd|j_nd|j_|jjdk(rIt}|j dk(r&|| j|j} n|| |} n|jjdk(r=t}|| jd|j |jd} n,|jjdk(rt}|| |} t| | | j | j"S) a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoProcessor, AutoModelForSequenceClassification >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> model = AutoModelForSequenceClassification.from_pretrained("microsoft/markuplm-base", num_labels=7) >>> html_string = " Page Title " >>> encoding = processor(html_string, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> loss = outputs.loss >>> logits = outputs.logits ```r4rN regressionsingle_label_classificationmulti_label_classificationr=rM)r r+rIr8 problem_typer/rirArlrtrr:rrrrrr)r7rwrCrDrrrWrnrKrr=rr>r6rAs r;rIz)MarkupLMForSequenceClassification.forward7sT $--   ))))%'       ]3 /  {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#FF3D))-JJ+- B @&++b/R))-II,./'!//))   rJrN)rKrLrMr#rrrArrrrrrIrOrPs@r;rPrP s  *..2.2.2.2,0-1&*T <<$&T  t+T  t+ T  t+ T  t+ T llT)T ||d*T  t#T +,T  u|| 7 7T T rJrP)r,rPrFrr )r)CrNcollections.abcrrArtorch.nnrrrr r activationsr modeling_layersr modeling_outputsr r rrrrmodeling_utilsrrprocessing_utilsr pytorch_utilsrutilsrrrr utils.genericrutils.output_capturingrconfiguration_markuplmr get_loggerrKloggerModulerrRrrrrrrrrfloatrrrrrr rr,rFrP__all__rJr;rjsL$ AA&!9G&6RR752   H %/ bii/ dpph   299  RYY  RYY  bii$ ryy "!"))!$% II% <<% % << % LL4' %  %%.3)BII3)n ..D bii 2ioii$f +f f Rb #:b b J V %<V  V r g (?g  g T rJ