^jԟzdZddlmZddlZddlmZddlmZmZmZddl m Z ddl m Z dd lmZdd lmZmZmZmZmZmZdd lmZmZdd lmZdd lmZddlmZm Z m!Z!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(e"jRe*Z+ejXZ-Gddej\Z/ dCdej\dej`dej`dej`dej`dzde1de1fdZ2Gddej\Z3Gdd ej\Z4Gd!d"ej\Z5Gd#d$ej\Z6Gd%d&ej\Z7Gd'd(eZ8Gd)d*ej\Z9Gd+d,ej\Z:Gd-d.ej\Z;Gd/d0ej\Z<Gd1d2ej\Z=e Gd3d4eZ>e Gd5d6e>Z?e Gd7d8e>Z@e d9:Gd;dZAe d=:Gd>d?e>ZBe Gd@dAe>ZCgdBZDy)DzPyTorch LayoutLM model.)CallableN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingMaskedLMOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)apply_chunking_to_forward)TransformersKwargsauto_docstringcan_return_tuplelogging)merge_with_config_defaults)capture_outputs)LayoutLMConfigc4eZdZdZfdZ ddZxZS)LayoutLMEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.ct|tj|j|j |j |_tj|j|j |_ tj|j|j |_ tj|j|j |_ tj|j|j |_ tj|j|j |_tj|j|j |_t#|j |j$|_tj(|j*|_|j/dt1j2|jj5ddy)N) padding_idxeps position_idsrF) persistent)super__init__r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsmax_2d_position_embeddingsx_position_embeddingsy_position_embeddingsh_position_embeddingsw_position_embeddingstype_vocab_sizetoken_type_embeddingsLayoutLMLayerNormlayer_norm_eps LayerNormDropouthidden_dropout_probdropoutregister_buffertorcharangeexpandselfconfig __class__s y/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/layoutlm/modeling_layoutlm.pyr)zLayoutLMEmbeddings.__init__3s[ !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2S2SU[UgUg%h"%'\\&2S2SU[UgUg%h"%'\\&2S2SU[UgUg%h"%'\\&2S2SU[UgUg%h"%'\\&2H2H&J\J\%]"*6+=+=6CXCXYzz&"<"<=   ELL)G)GHOOPWXej  c||j}n|jdd}|d}| |jn |j}||jddd|f}|&tj|tj |}||j |}|} |j|} |j|dddddf} |j|dddddf} |j|dddddf} |j|dddddf}|j|dddddf|dddddfz }|j|dddddf|dddddfz }|j|}| | z| z| z| z|z|z|z|z}|j|}|j|}|S#t$r}td|d}~wwxYw)Nr&rdtypedevicerrz:The `bbox`coordinate values should be within 0-1000 range.)sizerKr$r?zeroslongr.r0r2r3 IndexErrorr4r5r7r:r=)rC input_idsbboxtoken_type_idsr$ inputs_embeds input_shape seq_lengthrKwords_embeddingsr0left_position_embeddingsupper_position_embeddingsright_position_embeddingslower_position_embeddingser4r5r7 embeddingss rFforwardzLayoutLMEmbeddings.forwardDs1  #..*K',,.s3K ^ %.%:!! @T@T  ,,Q  ^ J ! "& '( ((  ( (  ( $  $$ $$ $ ^^J/ \\*- ) bYZ`a a bs,A,F77 G G  G)NNNNN)__name__ __module__ __qualname____doc__r)r^ __classcell__rEs@rFrr0s!Q &  5rGrmodulequerykeyvalueattention_maskscalingr=c tj||jdd|z}|||z}tjj |dtj j|j}tjj|||j}tj||} | jddj} | |fS)NrLrr&)dimrJ)ptrainingr) r?matmul transposer functionalsoftmaxfloat32torJr=rn contiguous) rerfrgrhrirjr=kwargs attn_weights attn_outputs rFeager_attention_forwardry}s<<s}}Q':;gEL!#n4 ==((2U]](SVVW\WbWbcL==((6??([L,,|U3K''1-88:K  $$rGc eZdZfdZ ddej dej dzdeede ej ej dzffdZ xZ S) LayoutLMSelfAttentionc$t||j|jzdk7r2t |ds&t d|jd|jd||_|j|_t|j|jz |_|j|jz|_ tj|j|j|_ tj|j|j|_ tj|j|j|_tj|j |_|j |_|jdz|_y)Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()g)r(r)r,num_attention_headshasattr ValueErrorrDintattention_head_size all_head_sizerLinearrfrgrhr;attention_probs_dropout_probr=attention_dropoutrjrBs rFr)zLayoutLMSelfAttention.__init__sC     : : :a ?PVXhHi#F$6$6#78 445Q8   #)#=#= #&v'9'9F $))#2. CCbC$*B*BC zz-055lCMMaQRS XXm,11,?II!QO zz-055lCMMaQRS (?(M(M KK , ,.E) %8      % $}}C$2H2HLL %  % ! \*k));;;;FFH L((rGN) r_r`rar)r?Tensor FloatTensorrrtupler^rcrds@rFr{r{sd6048)||)))D0)+, ) u||U\\D00 1 )rGr{cneZdZfdZdej dej dej fdZxZS)LayoutLMSelfOutputc(t|tj|j|j|_tj |j|j|_tj|j|_ yNr") r(r)rrr,denser:r9r;r<r=rBs rFr)zLayoutLMSelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= rGr input_tensorrcr|j|}|j|}|j||z}|Srrr=r:rCrrs rFr^zLayoutLMSelfOutput.forward7 =1  ]3 }|'CD rGr_r`rar)r?rr^rcrds@rFrr1> U\\RWR^R^rGrc eZdZfdZ ddej dej dzdeedej fdZ xZ S) LayoutLMAttentioncbt|t||_t ||_yr)r(r)r{rCroutputrBs rFr)zLayoutLMAttention.__init__s& )&1 (0 rGNrrirvrc ^|}|j|fd|i|\}}|j||}|SNri)rCr)rCrrirvresidual_s rFr^zLayoutLMAttention.forwardsK !$499  )   q  M8< rGr) r_r`rar)r?rrrrr^rcrds@rFrrsQ148 || ))D0 +,   rGrcVeZdZfdZdej dej fdZxZS)LayoutLMIntermediatect|tj|j|j |_t|jtrt|j|_ y|j|_ yr) r(r)rrr,intermediate_sizer isinstance hidden_actstrr intermediate_act_fnrBs rFr)zLayoutLMIntermediate.__init__s] YYv1163K3KL f'' -'-f.?.?'@D $'-'8'8D $rGrrcJ|j|}|j|}|Sr)rrrCrs rFr^zLayoutLMIntermediate.forwards& =1 00? rGrrds@rFrrs#9U\\ellrGrcneZdZfdZdej dej dej fdZxZS)LayoutLMOutputc(t|tj|j|j |_tj|j |j|_tj|j|_ yr) r(r)rrrr,rr:r9r;r<r=rBs rFr)zLayoutLMOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= rGrrrcr|j|}|j|}|j||z}|Srrrs rFr^zLayoutLMOutput.forwardrrGrrds@rFrrrrGrc eZdZfdZ d dej dej dzdeedej fdZ dZ xZ S) LayoutLMLayerct||j|_d|_t ||_t ||_t||_ y)Nr) r(r)chunk_size_feed_forward seq_len_dimr attentionr intermediaterrrBs rFr)zLayoutLMLayer.__init__sI '-'E'E$*6208$V, rGNrrirvrc |j|fd|i|}t|j|j|j|}|Sr)rrfeed_forward_chunkrr)rCrrirvs rFr^zLayoutLMLayer.forwardsY '  )  2  # #T%A%A4CSCSUb rGcL|j|}|j||}|Sr)rr)rCattention_outputintermediate_output layer_outputs rFrz LayoutLMLayer.feed_forward_chunk+s,"//0@A{{#68HI rGr) r_r`rar)r?rrrrr^rrcrds@rFrrsV-48||))D0+,   $rGrc neZdZfdZ ddej dej dzdeede fdZ xZ S) LayoutLMEncoderct|||_tjt |j Dcgc] }t|c}|_d|_ ycc}w)NF) r(r)rDr ModuleListrangenum_hidden_layersrlayergradient_checkpointing)rCrDirEs rFr)zLayoutLMEncoder.__init__3sN  ]]5IaIaCb#caM&$9#cd &+#$dsA#Nrrirvrc P|jD] }|||fi|}t|S)N)last_hidden_state)rr )rCrrirv layer_modules rFr^zLayoutLMEncoder.forward9sC !JJ L(M +  rGr) r_r`rar)r?rrrrr r^rcrds@rFrr2sM,48 || ))D0 +,    rGrcVeZdZfdZdej dej fdZxZS)LayoutLMPoolerct|tj|j|j|_tj |_yr)r(r)rrr,rTanh activationrBs rFr)zLayoutLMPooler.__init__Ms9 YYv1163E3EF '')rGrrc\|dddf}|j|}|j|}|S)Nr)rr)rCrfirst_token_tensor pooled_outputs rFr^zLayoutLMPooler.forwardRs6+1a40 #56  6 rGrrds@rFrrLs#$ U\\ellrGrcVeZdZfdZdej dej fdZxZS)LayoutLMPredictionHeadTransformcht|tj|j|j|_t |jtrt|j|_ n|j|_ tj|j|j|_ yr) r(r)rrr,rrrrr transform_act_fnr:r9rBs rFr)z(LayoutLMPredictionHeadTransform.__init__]s{ YYv1163E3EF f'' -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STrGrrcl|j|}|j|}|j|}|Sr)rrr:rs rFr^z'LayoutLMPredictionHeadTransform.forwardfs4 =1 --m< }5 rGrrds@rFrr\s$UU\\ellrGrc$eZdZfdZdZxZS)LayoutLMLMPredictionHeadct|t||_t j |j |jd|_t jtj|j|_ y)NT)bias) r(r)r transformrrr,r+decoder Parameterr?rNrrBs rFr)z!LayoutLMLMPredictionHead.__init__os[ 8@yy!3!3V5F5FTR LLV->->!?@ rGcJ|j|}|j|}|Sr)rrrs rFr^z LayoutLMLMPredictionHead.forwardxs$}5  ]3 rG)r_r`rar)r^rcrds@rFrrnsArGrcVeZdZfdZdej dej fdZxZS)LayoutLMOnlyMLMHeadcBt|t||_yr)r(r)r predictionsrBs rFr)zLayoutLMOnlyMLMHead.__init__s 3F;rGsequence_outputrc(|j|}|Sr)r)rCrprediction_scoress rFr^zLayoutLMOnlyMLMHead.forwards ,,_=  rGrrds@rFrrs#<!u||! !rGrcdeZdZUeed<dZdZeedZ e jfdZ xZ S)LayoutLMPreTrainedModelrDlayoutlmT)r attentionscXt||t|tr t j |j yt|trZt j|jtj|jjdjdyy)zInitialize the weightsr&r%N)r( _init_weightsrrinitzeros_rrcopy_r$r?r@rrA)rCrerEs rFrz%LayoutLMPreTrainedModel._init_weightssx f% f6 7 KK $  2 3 JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h4rG)r_r`rar__annotations__base_model_prefixsupports_gradient_checkpointingrr{_can_record_outputsr?no_gradrrcrds@rFrrsA "&*#&+ U]]_iirGrc&eZdZfdZdZdZeee dde jdzde jdzde jdzde jdzd e jdzd e jdzd e e d eezfd ZxZS) LayoutLMModelct||||_t||_t ||_t||_|jyr) r(r)rDrr]rencoderrpooler post_initrBs rFr)zLayoutLMModel.__init__sG   ,V4&v. $V,  rGc.|jjSrr]r.rCs rFget_input_embeddingsz"LayoutLMModel.get_input_embeddingss...rGc&||j_yrr)rCrhs rFset_input_embeddingsz"LayoutLMModel.set_input_embeddingss*/'rGNrQrRrirSr$rTrvrc (| | td|#|j|||j}n!||jdd}n td| |jn |j} |t j || }|&t j |tj| }|)t j |dztj| }|jdjd } | j|j } d | z t j|jjz} |j||||| } |j| | fi|} | d } |j| }t!| |S)a bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMModel >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMModel.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> outputs = model( ... input_ids=input_ids, bbox=bbox, attention_mask=attention_mask, token_type_ids=token_type_ids ... ) >>> last_hidden_states = outputs.last_hidden_state ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer&z5You have to specify either input_ids or inputs_embeds)rKrI)rrL)rJg?)rQrRr$rSrTr)r pooler_output)r%warn_if_padding_and_no_attention_maskrMrKr?onesrNrO unsqueezertrJfinfominr]rrr )rCrQrRrirSr$rTrvrUrKextended_attention_maskembedding_outputencoder_outputsrrs rFr^zLayoutLMModel.forwardsf  ]%>cd d  "  6 6y. Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ FCN  !"[[EJJvVN <;;{T1FSD"0":":1"="G"G"J"9"<"<4::"<"N#&)@#@EKKPTPZPZD[D_D_"_??%)' + '$,,  #   *!, O4 )-'  rG)NNNNNN)r_r`rar)rr rrrr? LongTensorrrrrr r^rcrds@rFrrs /0 .2(,37260426[ ##d*[ %[ ))D0 [ ((4/ [ &&- [ ((4/[ +,[  + +[  [ rGrcLeZdZdddZfdZdZdZdZee dd e jdzd e jdzd e jdzd e jdzd e jdzde jdzde jdzde edeezfdZxZS)LayoutLMForMaskedLMzcls.predictions.biasz*layoutlm.embeddings.word_embeddings.weight)zcls.predictions.decoder.biaszcls.predictions.decoder.weightct||t||_t ||_|j yr)r(r)rrrclsrrBs rFr)zLayoutLMForMaskedLM.__init__s4  %f- &v. rGcB|jjjSrrr]r.rs rFrz(LayoutLMForMaskedLM.get_input_embeddings!}}''777rGcB|jjjSr)rrrrs rFget_output_embeddingsz)LayoutLMForMaskedLM.get_output_embeddings$sxx##+++rGc||jj_|j|jj_yr)rrrr)rCnew_embeddingss rFset_output_embeddingsz)LayoutLMForMaskedLM.set_output_embeddings's,'5$$2$7$7!rGNrQrRrirSr$rTlabelsrvrc :|j||f||||d|} | d} |j| } d} |Ft} | | jd|jj |jd} t | | | j| jS)a2 bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForMaskedLM >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForMaskedLM.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "[MASK]"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> labels = tokenizer("Hello world", return_tensors="pt")["input_ids"] >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=labels, ... ) >>> loss = outputs.loss ```)rirSr$rTrNr&losslogitsrr) rrrrrDr+rrr)rCrQrRrirSr$rTr#rvoutputsrrmasked_lm_lossloss_fcts rFr^zLayoutLMForMaskedLM.forward+sz $--   *)%'   "!* HH_5  ')H%!&&r4;;+A+AB BN $!//))   rGNNNNNNN)r_r`ra_tied_weights_keysr)rrr"rrr?rrrrrrr^rcrds@rFrrs)?*V 8,8.2(,37260426*.U ##d*U %U ))D0 U ((4/ U &&- U ((4/U   4'U +,U   U U rGrz LayoutLM Model with a sequence classification head on top (a linear layer on top of the pooled output) e.g. for document image classification tasks such as the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset. ) custom_introc6eZdZfdZdZee ddejdzdejdzdejdzdejdzdejdzd ejdzd ejdzd e e d e e zfd ZxZS)!LayoutLMForSequenceClassificationc,t|||j|_t||_t j |j|_t j|j|j|_ |jyr r(r) num_labelsrrrr;r<r=rr, classifierrrBs rFr)z*LayoutLMForSequenceClassification.__init__i   ++%f- zz&"<"<= ))F$6$68I8IJ rGcB|jjjSrrrs rFrz6LayoutLMForSequenceClassification.get_input_embeddingsrrGNrQrRrirSr$rTr#rvrc |jd ||||||d|} | d} |j| } |j| } d} ||jj|j dk(rd|j_nl|j dkDrL|j tjk(s|j tjk(rd|j_nd|j_|jjdk(rIt} |j dk(r&| | j|j} n| | |} n|jjdk(r=t} | | jd|j |jd} n,|jjdk(rt} | | |} t| | | j | j"S) aB bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForSequenceClassification >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForSequenceClassification.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> sequence_label = torch.tensor([1]) >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=sequence_label, ... ) >>> loss = outputs.loss >>> logits = outputs.logits ```rQrRrirSr$rTrN regressionsingle_label_classificationmulti_label_classificationr&r%)rr=r3rD problem_typer2rJr?rOrrsqueezerrrrrr)rCrQrRrirSr$rTr#rvr(rr'r&r*s rFr^z)LayoutLMForSequenceClassification.forwardsz $-- ))%'      ]3 /  {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#FF3D))-JJ+- B @&++b/R))-II,./'!//))   rGr+)r_r`rar)rrrr?rrrrrrr^rcrds@rFr/r/s8.2(,37260426*.f ##d*f %f ))D0 f ((4/ f &&- f ((4/f   4'f +,f  ) )f f rGr/a3 LayoutLM Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for sequence labeling (information extraction) tasks such as the [FUNSD](https://guillaumejaume.github.io/FUNSD/) dataset and the [SROIE](https://rrc.cvc.uab.es/?ch=13) dataset. c6eZdZfdZdZee ddejdzdejdzdejdzdejdzdejdzd ejdzd ejdzd e e d e e zfd ZxZS)LayoutLMForTokenClassificationc,t|||j|_t||_t j |j|_t j|j|j|_ |jyrr1rBs rFr)z'LayoutLMForTokenClassification.__init__ r4rGcB|jjjSrrrs rFrz3LayoutLMForTokenClassification.get_input_embeddingsrrGNrQrRrirSr$rTr#rvrc H|jd||||||d|} | d} |j| } |j| } d} |>> from transformers import AutoTokenizer, LayoutLMForTokenClassification >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForTokenClassification.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> token_labels = torch.tensor([1, 1, 0, 0]).unsqueeze(0) # batch size of 1 >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=token_labels, ... ) >>> loss = outputs.loss >>> logits = outputs.logits ```r7rNr&r%r;) rr=r3rrr2rrr)rCrQrRrirSr$rTr#rvr(rr'r&r*s rFr^z&LayoutLMForTokenClassification.forwardsv $-- ))%'   "!*,,71  ')HFKKDOO>> from transformers import AutoTokenizer, LayoutLMForQuestionAnswering >>> from datasets import load_dataset >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("impira/layoutlm-document-qa", add_prefix_space=True) >>> model = LayoutLMForQuestionAnswering.from_pretrained("impira/layoutlm-document-qa", revision="1e3ebac") >>> dataset = load_dataset("nielsr/funsd", split="train") >>> example = dataset[0] >>> question = "what's his name?" >>> words = example["words"] >>> boxes = example["bboxes"] >>> encoding = tokenizer( ... question.split(), words, is_split_into_words=True, return_token_type_ids=True, return_tensors="pt" ... ) >>> bbox = [] >>> for i, s, w in zip(encoding.input_ids[0], encoding.sequence_ids(0), encoding.word_ids(0)): ... if s == 1: ... bbox.append(boxes[w]) ... elif i == tokenizer.sep_token_id: ... bbox.append([1000] * 4) ... else: ... bbox.append([0] * 4) >>> encoding["bbox"] = torch.tensor([bbox]) >>> word_ids = encoding.word_ids(0) >>> outputs = model(**encoding) >>> loss = outputs.loss >>> start_scores = outputs.start_logits >>> end_scores = outputs.end_logits >>> start, end = word_ids[start_scores.argmax(-1)], word_ids[end_scores.argmax(-1)] >>> print(" ".join(words[start : end + 1])) M. Hamann P. Harper, P. Martinez ```r7rrr&)rlN) ignore_indexrL)r& start_logits end_logitsrrr;) rrFsplitr=rulenrMclamprrrr)rCrQrRrirSr$rTrIrJrvr(rr'rMrN total_loss ignored_indexr* start_lossend_losss rFr^z$LayoutLMForQuestionAnswering.forwards~ $-- ))%'   "!*1#)<<r<#: j#++B/::< ''+668   &=+D?'')*Q."1"9"9""==%%'(1, - 5 5b 9 (--a0M-33A}EO)//=AM']CH!,@J M:H$x/14J+%!!//))   rG)T)NNNNNNNN)r_r`rar)rrrr?rrrrrrr^rcrds@rFrDrDps 8.2(,372604263715e ##d*e %e ))D0 e ((4/ e &&- e ((4/e ))D0e ''$.e +,e  - -e e rGrD)rr/r?rDrr)r)Erbcollections.abcrr?rtorch.nnrrrr r activationsr modeling_layersr modeling_outputsr r rrrrmodeling_utilsrrprocessing_utilsr pytorch_utilsrutilsrrrr utils.genericrutils.output_capturingrconfiguration_layoutlmr get_loggerr_loggerr:r8Modulerrfloatryr{rrrrrrrrrrrrrr/r?rD__all__r;rGrFrhsg$ AA&!9G&6RR752   H %LLIIh% II% <<% % << % LL4' %  %%.3)BII3)n   . 299  RYY .D bii 4 RYY  bii$ ryy "!"))!ioii&p +p p fp 1p p f v (?v  v rb %<b b Jy #:y y x rG