^jȖdZddlmZddlZddlmZmZddlmZddl m Z m Z ddl m Z dd lmZmZmZmZmZdd lmZdd lmZdd lmZdd lmZmZmZddlmZm Z ddl!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-ddl.m/Z/edeGddeZ0Gdde,Z1Gdde)Z2GddejfZ4Gd d!e(Z5Gd"d#e+Z6Gd$d%e&Z7Gd&d'e*Z8eGd(d)e-Z9eGd*d+e9Z:Gd,d-ejfZ;ed.Gd/d0e9Z<ed1Gd2d3e9Z=Gd4d5e$Z>Gd6d7ejfZ?Gd8d9ejfZ@Gd:d;ejfZAGd<d=ejfZBGd>d?ejfZCGd@dAejfZDeGdBdCe9ZEedDGdEdFe e9ZFgdGZGy)HzPyTorch BEiT model.) dataclassN)Tensornn)initialization) BackboneMixinfilter_output_hidden_states)create_bidirectional_mask)BackboneOutputBaseModelOutputWithPoolingImageClassifierOutputMaskedLMOutputSemanticSegmenterOutput)PreTrainedModel)Unpack)#compile_compatible_method_lru_cache)TransformersKwargsauto_docstring torch_int)can_return_tuplemerge_with_config_defaults)capture_outputs)ResNetConvLayer) SwinDropPath) ViTAttention ViTEmbeddingsViTLayerViTMLPViTPatchEmbeddingsViTPreTrainedModel) BeitConfigz- Class for outputs of [`BeitModel`]. ) custom_introceZdZdZy)BeitModelOutputWithPoolingaF pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Average of the last layer hidden states of the patch tokens (excluding the *[CLS]* token) if *config.use_mean_pooling* is set to True. If set to False, then the final hidden state of the *[CLS]* token will be returned. N)__name__ __module__ __qualname____doc__p/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/beit/modular_beit.pyr&r&+sr,r&c eZdZy)BeitPatchEmbeddingsNr'r(r)r+r,r-r/r/:r,r/cveZdZdeddfdZ ddej dejdzdej fdZy) BeitEmbeddingsconfigreturnNctjj|tjt j dd|j |_|jr4tjt j dd|j nd|_ t||_ |j|_ |jj}|jr7tjt j d|dz|j nd|_tj |j"|_y)Nr")rModule__init__ Parametertorchzeros hidden_size cls_tokenuse_mask_token mask_tokenr/patch_embeddings patch_size num_patches use_absolute_position_embeddingsposition_embeddingsDropouthidden_dropout_probdropout)selfr4rBs r-r8zBeitEmbeddings.__init__?s 4 ekk!Q8J8J&KLQWQfQf",,u{{1a9K9K'LMlp 3F ; ++++77 66 LLQ a9K9KL M  zz&"<"<= r, pixel_valuesbool_masked_posc|j\}}}}|j|}|j\}}}|K|jj ||d} |j dj | } |d| z z| | zz}|jj |dd} tj| |fd}|j||j|||z}|j|}|SNr"dim) shaper@sizer?expand unsqueezetype_asr=r:catrDinterpolate_pos_encodingrG) rHrIrJ_heightwidth embeddings batch_sizeseq_len mask_tokensmask cls_tokenss r-forwardzBeitEmbeddings.forwardNs +001fe**<8 !+!2 GQ  &//00WbIK",,R088ED#q4x0;3EEJ^^**:r2> YY J7Q?  # # /#d&C&CJPVX]&^^J\\*- r,N) r'r(r)r#r8r:r BoolTensorr`r+r,r-r3r3>sN >z >d >$48ll))D0  r,r3ceZdZdeddffd Zeeddeeefde jfdZ d d e de jfd Z xZS) BeitRelativePositionBiasr4r5Nct||j}t|tt fs||f}|d|j z|d|j zf|_d|jdzdz d|jdzdz zdz|_tjtj|j|j|_yNrr"rr)superr8 image_size isinstancetuplelistrA window_sizenum_relative_distancerr9r:r;num_attention_headsrelative_position_bias_table)rHr4rh __class__s r-r8z!BeitRelativePositionBias.__init__is && *udm4$j1J&qMV->->> 1 QWQbQb@bc&'$*:*:1*=&=&Aa$JZJZ[\J]F]`aFa%bef%f",.LL KK22F4N4N O- )r, )maxsizerlc d|dzdz d|dzdz zdz}|d|dz}tjtjtjtj|dtj|ddd}|dddddf|dddddfz j dddj }|dddddfxx|ddz z cc<|dddddfxx|ddz z cc<|dddddfxxd|dzdz zcc<tj|dzfdz|j }|jd |ddddf<|dz |dddf<|dz |dddf<|dz |d <|S) z This method creates the relative position index, modified to support arbitrary window sizes, as introduced in [MiDaS v3.1](https://huggingface.co/papers/2307.14460). rrr"rij)indexing) start_dimN)rQdtyperM)rr) r:flattenstackmeshgridarangepermute contiguousr;rwsum)rlrm window_areacoords_flattenrelative_coordsrelative_position_indexs r- generate_relative_position_indexz9BeitRelativePositionBias.generate_relative_position_indexus"#[^!3a!7A A)B12&)>)BA&(=(A%&&r,rVcd|jdzdz }d|jdzdz }d|dzdz }d|dzdz }|j}|j} ||zdz} |d| dz } | jd||dj dddd} t j j| t|t|fd} | j ddddj| dz d} tj| || dz dg} |j|}| |jd}|j|d|dzdz|d|dzdzd}|j dddj}|rCt j j|jd||fdd jd}|jdS) zu Modification of timm.models.beit.py: Attention._get_rel_pos_bias to support arbitrary window sizes. rrr"rNrMbilinear)rQmodeFrQr align_corners)rlrormreshaper|r functional interpolaterr:rUrviewr}rSsqueeze)rHrlrVdim_size old_height old_width new_height new_width old_relative_position_bias_tableold_num_relative_distancenew_num_relative_distance old_sub_table new_sub_table new_relative_position_bias_tablerrelative_position_biass r-r`z BeitRelativePositionBias.forwards-))!,,q0 ((++a/ Q'!+  A&* +/+L+L($($>$>!$.$:Q$>!89X;TWX;XY %--aJKSSTUWXZ[]^_  11 :!6 )8L MT^2 &--aAq9AAB[^_B_acd +099 <=VYZ=Z=\] ^, (#'"G"G "T!ABYB^B^_aBb!c"8!%>&003)# &?& gaj #&//22r,)FN)r'r(r)r#r8 staticmethodrrjintr:rrboolr` __classcell__rps@r-rdrdhsk  z  d  (4'eCHo'%,,'5'4-3T-3]b]i]i-3r,rdc$eZdZdeffd ZxZS) BeitAttentionr4ct||tj|j|j |j z|_tj|j|j |j zd|_tj|j|j |j z|_ tj|j |j z|j|_ y)NF)bias) rgr8rLinearr<rnhead_dimq_projk_projv_projo_projrHr4rps r-r8zBeitAttention.__init__s  ii 2 2F4N4NQUQ^Q^4^_ ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^_ ii : :T]] JFL^L^_ r,)r'r(r)r#r8rrs@r-rrs`z``r,rc eZdZy)BeitMLPNr0r+r,r-rrr1r,rc eZdZy) BeitDropPathNr0r+r,r-rrr1r,rc eZdZdZd dedeffd Z ddejdejdzde d e e e fdzd e e d ejf d ZxZS) BeitLayerz?This corresponds to the Block class in the timm implementation.r4drop_path_ratect||j|_|dkDr t|nt j |_|j}|dkDr7t j|tj|jzdnd|_ |dkDr7t j|tj|jzdnd|_ |jrt||_yd|_y)NrT) requires_gradg?)rgr8rArrIdentity drop_pathlayer_scale_init_valuer9r:onesr<lambda_1lambda_2use_relative_position_biasrdr)rHr4r init_valuesrps r-r8zBeitLayer.__init__s  ++9G#9Mn5SUS^S^S`33 ^ilm^mBLLuzz&2D2D'EEUY Zsv _jlm^mBLLuzz&2D2D'EEUY Zsv KQJkJk&>v&F#qu#r,N hidden_statesattention_maskrV resolutionkwargsr5c &|jM|\}}||jz||jzf}|j|||jd} || |zn| }|} |j|}|j|fd|i|\}} |j |}|j |z}|j|| z}|} |j|}|j|}|j |}|j|z}|j|| z}|S)Nr")rr) rrArPlayernorm_before attentionrGrrlayernorm_aftermlpr) rHrrrVrrrXrYrlrresidualrWs r-r`zBeitLayer.forwardsE  & & 2&MFE!T__4et6NOK%)%@%@5 @S@STU@V&A& ">  )   q  ]3  5 }5@ !,,]; /  ]3  5 }5@ r,)rNFN)r'r(r)r*r#floatr8r:rrrjrrrr`rrs@r-rrsI vz v5 v"/3).-1 &||& t+&#' & #s(Od* & +, & &r,rc&eZdZdgZdgZdZdZdZy)BeitPreTrainedModelrz.*relative_position_index.*Fctj||t|trwt j |j |jt j |j|j t j |jyyt|tr t j |jyt|trt|jtjrit j|j|j j"t j|j$|j j"yyy)zInitialize the weightsN)r _init_weightsrir3initzeros_r=r?rDrdrorrrr9 constant_r4rr)rHmodules r-rz!BeitPreTrainedModel._init_weightss%%dF3 fn - KK(( )  , F--.))5 F6676  8 9 KK;; <  *&//2<<8v 0R0RSv 0R0RS9+r,N)r'r(r)_no_split_modules"_keys_to_ignore_on_load_unexpected_supports_flash_attn_supports_flex_attnrr+r,r-rr s%$ *H)I& Tr,rceZdZddededdffd Zeede dde jd e jdzd ed e jdzd e e def d ZxZS) BeitModelr4add_pooling_layerr5Nc t||||_t||_|j r t |nd|_t|jDcgc]+}|j|zt|jdz dz -}}tj|Dcgc]}t||c}|_|j rtj"n*tj$|j&|j(|_|r t-|nd|_|j1ycc}wcc}w)zv add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer Nr")reps)rgr8r4r3rZ!use_shared_relative_position_biasrdshared_position_biasrangenum_hidden_layersrmaxr ModuleListrlayersuse_mean_poolingr LayerNormr<layer_norm_eps layernorm BeitPoolerpooler post_init)rHr4ridrop_path_ratesrrps r-r8zBeitModel.__init__&s   (0060X0X $V ,^b !W\\b\t\tVu QRF ! !A %F,D,Dq,H!(L L  mmRa$bQYva%H$bc $44BKKM",,vGYGY_e_t_t:u ->j(4   %cs 0D7"D<F)tie_last_hidden_statesrIrJrVrrc |j||}|jdd}t|j||}|ja|\}} ||jj z| |jj zf} |j | ||jd} || |zn| }|} |j D]} | | f|||d|} |j| }|j|j|nd}t||S) z bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`, *optional*): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). )rJrN)r4 inputs_embedsrr")rVr)rrVr)last_hidden_state pooler_output) rZrPr r4rrArrrr&)rHrIrJrVrrembedding_outputrrXrYrlshared_relative_position_biasrlayersequence_output pooled_outputs r-r`zBeitModel.forward?s< ??<?Y!''+ 2;;*)   $ $ 0&MFE!T[[%;%;;UdkkF\F\=\]K,0,E,E6NYiYoYopqYr-F- ) "-.>2  ) [[ E!-)A%   M ..78< 8O O4UY )O[hiir,)Tr)r'r(r)r#rr8rrrr:rrbrrr&r`rrs@r-rr$szdd2 E248)..2 -jll-j))D0-j#' -j  t+ -j +, -j $-j3 -jr,rc`eZdZdeddffd Zdej dej fdZxZS)rr4r5Nct||jr1tj|j |j |_yd|_y)Nr)rgr8rrrr<rrrs r-r8zBeitPooler.__init__ssA KQKbKbBLL++1F1F G hl r,rc|j,|j|ddddddfjdS|dddfS)Nr"r)rmeanrHrs r-r`zBeitPooler.forwardysDBF..B\t~~mAqr1H5::1=>ubopqstptbuur,) r'r(r)r#r8r:rr`rrs@r-rrrs4 z d vU\\vellvr,ra Beit Model transformer with a 'language' modeling head on top. BEiT does masked image modeling by predicting visual tokens of a Vector-Quantize Variational Autoencoder (VQ-VAE), whereas other vision models like ViT and DeiT predict RGB pixel values. As a result, this class is incompatible with [`AutoModelForMaskedImageModeling`], so you will need to use [`BeitForMaskedImageModeling`] directly if you wish to do masked image modeling with BEiT. ceZdZdeddffd ZdZee d dejdzdejdzdejdzd e d ejdzd e e deezfd ZxZS)BeitForMaskedImageModelingr4r5NcHt|||j|_t|d|_t j |j|j|_ t j|j|j|_ |jy)NFrr)rgr8 num_labelsrbeitrrr<rrr vocab_sizelm_headrrs r-r8z#BeitForMaskedImageModeling.__init__su   ++f> f&8&8f>S>STyy!3!3V5F5FG  r,cyrar+)rHs r-get_output_embeddingsz0BeitForMaskedImageModeling.get_output_embeddingssr,rIrJlabelsrVrrc ,|j|f|||d|}|j}|j|}|j|ddddf} d} | t j } | | ||} t | | |j|jS)a bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, BeitForMaskedImageModeling >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("microsoft/beit-base-patch16-224-pt22k") >>> model = BeitForMaskedImageModeling.from_pretrained("microsoft/beit-base-patch16-224-pt22k") >>> num_patches = (model.config.image_size // model.config.patch_size) ** 2 >>> pixel_values = image_processor(images=image, return_tensors="pt").pixel_values >>> # create random boolean mask of shape (batch_size, num_patches) >>> bool_masked_pos = torch.randint(low=0, high=2, size=(1, num_patches)).bool() >>> outputs = model(pixel_values, bool_masked_pos=bool_masked_pos) >>> loss, logits = outputs.loss, outputs.logits >>> list(logits.shape) [1, 196, 8192] ```)rJrVrNr"losslogitsr attentions) rrrrrCrossEntropyLossrrr ) rHrIrJrrVrroutputsrprediction_scoresmasked_lm_lossloss_fcts r-r`z"BeitForMaskedImageModeling.forwardsX$))  +%=)    "33..9 LLAB)?@  **,H%&7&H&QN$!//))   r,)NNNFN)r'r(r)r#r8rrrr:rrbrrrrjrr`rrs@r-rr~s z d -137&*)..2 @ llT)@ ))D0@  t# @ #' @  t+ @ +,@   @ @ r,rz Beit Model transformer with an image classification head on top (a linear layer on top of the average of the final hidden states of the patch tokens) e.g. for ImageNet. ceZdZdeddffd Zee d dejdzdejdzde de e de e zf d ZxZS) BeitForImageClassificationr4r5Nc.t|||j|_t|d|_|jdkDr*t j |j|jnt j|_ |jy)NTrr) rgr8rrrrrr<r classifierrrs r-r8z#BeitForImageClassification.__init__ss   ++f= OUN_N_bcNc"))F$6$68I8IJikititiv r,rIrrVrc |j|fd|i|}|j}|j|}d}||j|||j}t |||j |jS)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rVNr)rrr loss_functionr4r rr ) rHrIrrVrr rrrs r-r`z"BeitForImageClassification.forwards$))  %=    -- /  %%ffdkkBD$!//))   r,NNF)r'r(r)r#r8rrr:rrrrrjr r`rrs@r-rrs z d -1&*).  llT)   t#  #'  +,  & &    r,rceZdZ d dededeeeefzdedeeeefzezdedeeeefzded effd ZxZS) BeitConvLayer in_channels out_channels kernel_sizestridepaddingrdilationgroups activationc ft |tj|||||||||_y)N)rrrrrrrr)rgr8rConv2d convolution) rHrrrrrrrrrrps r-r8zBeitConvLayer.__init__s9 99#%#  r,)rr"rFr"r"relu) r'r(r)rrjstrrr8rrs@r-rrs .//0*+    5c?*    uS#X&,   c3h'    r,rcveZdZdedededdffd Zdej deeefdej fd ZxZ S) BeitPyramidPoolingBlock pool_scalerchannelsr5Nc|t|tj||_t ||d|_y)Nr"r)rgr8rAdaptiveAvgPool2dpoolingrconv)rHr'rr(rps r-r8z BeitPyramidPoolingBlock.__init__/s0 ++J7 !+xQG r,inputrQc|j|}|j|}tjj ||dd}|S)NrFr)r,r-rrr)rHr.rQ hidden_states r-r`zBeitPyramidPoolingBlock.forward4sB||E* yy. }}00Dzin0o r,) r'r(r)rr8r:rrjr`rrs@r-r&r&.sSH3HSHCHDH U\\sCxU\\r,r&c|eZdZdZdeedfdededdffd Zd ejde ejfd Z xZ S) BeitPyramidPoolingModuleak Pyramid Pooling Module (PPM) used in PSPNet. Args: pool_scales (tuple[int]): Pooling scales used in Pooling Pyramid Module. in_channels (int): Input channels. channels (int): Channels after modules, before conv_seg. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation. pool_scales.rr(r5Nc t|||_||_||_t j |Dcgc]}t|||c}|_ycc}w)N)r'rr() rgr8r3rr(rrr&blocks)rHr3rr(r'rps r-r8z!BeitPyramidPoolingModule.__init__Hs\ &&  mm#. (:;aij    sArcv|jdd}|jDcgc] }|||c}Scc}w)Nr)rQ)rQr5)rHr original_sizeblocks r-r`z BeitPyramidPoolingModule.forwardTs6%**,QR0 FJkkRUm-8RRRs6) r'r(r)r*rjrr8r:rrkr`rrs@r-r2r2;sV   E#s(O  #  QT  Y]  SU\\Sd5<<6HSr,r2ceZdZdZdeddffd ZdeejdejfdZ deejdejfd Z xZ S) BeitUperHeadz Unified Perceptual Parsing for Scene Understanding. This head is the implementation of [UPerNet](https://huggingface.co/papers/1807.10221). Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation. r4r5Nc t||j|_|jgdz|_|j|_t j|j |jd|_ t|j|jd|j |_ t|jdt|j|j zz|j dd|_t j|_t j|_|jddD]o}|j j%t||j d|j"j%t|j |j ddqtt|j|j z|j dd|_y)Nr"r*rMrrr)rgr8r3r<rr(rr!rrr2 psp_modulesrlenpsp_bottleneckr lateral_convs fpn_convsappendfpn_bottleneck)rHr4rrps r-r8zBeitUperHead.__init__as !--"../!3** ))DMM63D3DRST4      R MM  ,   R 3t'7'7#84==#H H MM   ]]_++CR0 iK    % %mK\]&^ _ NN ! !- t}}Z[ef"g h i,   !DMM 1 MM  r,rc|d}tj|g|j|d}|j|SrL)r:rUr>r@)rHrr0s r- psp_forwardzBeitUperHead.psp_forwardsA$R( yy,!P1A1A,1O!PVWX ""<00r,encoder_hidden_statesc6g}t|j|D]\}}|j|||j|j|t |}t |dz ddD]L}||dz j dd}||dz tjj|||ddz||dz <Ng}t |dz D])}|j|j|||+|j|dt |dz ddD];}tjj|||dj dddd||<=tj|d}|j|} |j| } | S) Nr"rrMrrFrrN)ziprArCrFr?rrPrrrrBr:rUrDr) rHrGlaterals lateral_convr0used_backbone_levelsr prev_shapefpn_outsoutputs r-r`zBeitUperHead.forwards*-d.@.@BW*X 8 &L, OOL6 7 8 (()>?@ #8}+a/B7 A!!a%..qr2J&q1uo 0I0I *:U1J1HQUO +a/0 ). Args: config (BeitConfig): Configuration. in_channels kernel_size (int): The kernel size for convs in the head. Default: 3. dilation (int): The dilation rate for convs in the head. Default: 1. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation. r4in_indexrrr5Nc 0t||j|_|j|_|j |_|j|_ ||_ |dz|z}tj|_ |jdkDr|jjt|j|j |||t!|jdz D]?}|jjt|j |j |||A|jr8t|j|j z|j ||dz|_tj$|j |j&d|_y)Nrr)rrrr"r=r*)rgr8r<rauxiliary_channelsr(auxiliary_num_convs num_convsauxiliary_concat_input concat_inputrRrrconvsrCrrconv_catr!rr)rHr4rRrr conv_paddingrWrps r-r8zBeitFCNHead.__init__sQ !--11 33"99  #q(H4 ]]_ >>A  JJ  $$dmmVbmu  4>>A-.  !!!  $/ ,!)     )  4==0$--[bmqrbrDM))DMM63D3DRSTr,rGc||j}|}|jD] }||} |jr(|jt j ||gd}|j |}|S)Nr"rN)rRrYrXrZr:rUr)rHrGrrr-s r-r`zBeitFCNHead.forwardsn(7 JJ 0D /M 0    MM%))X}4MST*UVM 6 r,)rrr") r'r(r)r*r#rrjr8rkr:rr`rrs@r-rQrQsu no!U !U,/!UBE!UUX[`adfiai[jUj!U !UFT%,,-?ELLr,rQc neZdZdZd dedededdffd Zdejdejfd ZxZ S) BeitFPNUpBlockuE4x upsampling block: ConvTranspose → BN → GELU → ConvTranspose.r<rrr5Nct|tj|||||_tj ||_tj|_tj|||||_ y)Nrr) rgr8rConvTranspose2dconv_transpose1 BatchNorm2d normalizationGELUrconv_transpose2)rHr<rrrps r-r8zBeitFPNUpBlock.__init__sb !11+{Xclrs^^K8'')!11+{Xclrsr,rc|j|}|j|}|j|}|j|}|Sra)rbrdrrfrs r-r`zBeitFPNUpBlock.forwardsF,,]; **=9  6 ,,]; r,)rr) r'r(r)r*rr8r:rr`rrs@r-r^r^sHOtCtctstSWtU\\ellr,r^cteZdZdZdeffd ZdeejdfdeejdffdZ xZ S) BeitFPNNeckz 4-level feature pyramid neck for BeiT. Produces x4 upsample, x2 upsample, identity, and x2 downsample outputs from the four selected ViT feature maps. r4ct|t|j|_t j |j|jdd|_t jdd|_ y)Nrr`) rgr8r^r<fpn1rrafpn2 MaxPool2dfpn4rs r-r8zBeitFPNNeck.__init__sX "6#5#56 &&v'9'96;M;M[\efg LLQq9 r, feature_maps.r5c|j|d|j|d|d|j|dfSrf)rkrlrn)rHros r-r`zBeitFPNNeck.forwardsC IIl1o & IIl1o & O IIl1o &   r,) r'r(r)r*r#r8rjr:rr`rrs@r-ririsD :z:  E%,,*;$< u||UXGXAY r,riceZdZdeddffd Zeee d dejdzdejdzde de e de ezf d ZxZS) BeitForSemanticSegmentationr4r5Nc`t|||j|_t|d|_t |j jdk7r tdt||_ t||_ |jr t|nd|_|j!y)NFrr<zBeitForSemanticSegmentation requires config.out_indices to be a list of 4 integers, specifying which features to use from the backbone. One can use [3, 5, 7, 11] in case of a base-sized architecture.)rgr8rrrr?r4 out_indices ValueErrorrifpnr: decode_headuse_auxiliary_headrQauxiliary_headrrs r-r8z$BeitForSemanticSegmentation.__init__s   ++f> t{{&& '1 ,-  v&(/5;5N5Nk&1TX r,rIrrVrc  |$|jjdk(r td|j|fd|i|}|j|j \ }}}||jj z||jj zt fd|jjD} |j| } |j| } d} |j|j| } d} |>|j| ||jj| |jj} t| | |j|j S)aD labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, BeitForSemanticSegmentation >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("microsoft/beit-base-finetuned-ade-640-640") >>> model = BeitForSemanticSegmentation.from_pretrained("microsoft/beit-base-finetuned-ade-640-640") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> # logits are of shape (batch_size, num_labels, height, width) >>> logits = outputs.logits ```Nr"z/The number of labels should be greater than onerVc3K|]7}|dz ddddfjddjd9yw)r"NrrM) transposer).0rr[rG patch_height patch_widths r- z6BeitForSemanticSegmentation.forward..UsN  "!a% (AB / 9 9!Q ? G G TVXdfq r s=A) ignore_indexauxiliary_logitsauxiliary_loss_weightr)r4rrurrrPrArjrtrvrwryrsemantic_loss_ignore_indexrrr )rHrIrrVrr rWrXrYrorrrr[rGr~rs @@@@r-r`z#BeitForSemanticSegmentation.forward%shB  $++"8"8A"=NO O$))  %=   !( 5 5'3'9'9$ Avu!7!77 t{{555  [[,,  xx - !!,/    *#22<@   %%![[CC!1&*kk&G&G &D'!//))   r,r)r'r(r)r#r8rr rr:rrrrrjrr`rrs@r-rrrrszd* -1&*). G llT)G  t#G #' G +, G ( ( G !G r,rrzM BEiT backbone, to be used with frameworks like DETR and MaskFormer. c VeZdZfdZeeededee de fdZ xZ S) BeitBackbonec<t||t|jdzDcgc]}|jc}|_t |d|_|jr t|ntj|_ |jycc}w)Nr"Fr)rgr8rrr< num_featuresrradd_fpnrirrrvr)rHr4rWrps r-r8zBeitBackbone.__init__xst  9>v?W?WZ[?[9\]AV//]f> *0..;v&bkkm  ^sBrIrr5c *|j\}}}}||jjz}||jjz}|j|fi|} | j} d} t |j | D]d\} } | |jvs|jjr4| ddddddf} | jdd} | j|d||} | | fz } f|j| } t| | j| jS)a: Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> processor = AutoImageProcessor.from_pretrained("microsoft/beit-base-patch16-224") >>> model = AutoBackbone.from_pretrained( ... "microsoft/beit-base-patch16-224", out_features=["stage1", "stage2", "stage3", "stage4"] ... ) >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```r+Nr"rrM)rorr )rPr4rArrrI stage_names out_featuresreshape_hidden_statesr|rrvr r )rHrIrr[rWrXrYr~rr rrostager0s r-r`zBeitBackbone.forwards@(4'9'9$ Avu!7!77 t{{555 $))L3F3--  #&t'7'7#G 0 E<)));;44#/12q#9L#/#9#9!Q#?L#/#7#7 B Va#bL/  0xx - %!//))  r,) r'r(r)r8rr rrrrr r`rrs@r-rrrsN  3 3 +,3   3 !3 r,r)rrrrrrr)Hr* dataclassesrr:rrrrbackbone_utilsrr masking_utilsr modeling_outputsr r r rrmodeling_utilsrprocessing_utilsr pytorch_utilsrutilsrrr utils.genericrrutils.output_capturingrresnet.modeling_resnetrswin.modeling_swinrvit.modeling_vitrrrrr r!configuration_beitr#r&r/r3r7rdrrrrrrrrrrr&r2r:rQr^rirrr__all__r+r,r-rsF! &H6.&@BBI54-tt*  !;   , ']'TV3ryyV3r`L` f  < 77tT,TT.Jj#JjJjZ v vS !4S S l / !4/  / d O 4 bii SryyS