^jx dZddlZddlmZddlZddlmZddlmZddlm Z m Z ddl m Z dd l mZdd lmZmZmZmZmZdd lmZd d lmZer ddlmZmZndZdZedeGddeZedeGddeZedeGddeZGddej>Z Gddej>Z!Gdd ej>Z"Gd!d"ej>Z#Gd#d$ej>Z$Gd%d&ej>Z%Gd'd(ej>Z&Gd)d*ej>Z'Gd+d,ej>Z(Gd-d.ej>Z)Gd/d0ej>Z*Gd1d2ej>Z+eGd3d4eZ,eGd5d6e,Z-ed7Gd8d9e,Z.ed:Gd;dz9PyTorch Dilated Neighborhood Attention Transformer model.N) dataclass)nn)ACT2FN) BackboneMixinfilter_output_hidden_states)BackboneOutput)PreTrainedModel) ModelOutputOptionalDependencyNotAvailableauto_docstringis_natten_availablerequires_backends)can_return_tuple) DinatConfig) natten2dav natten2dqkrpbctNr argskwargss s/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/dinat/modeling_dinat.pyrr) ,..ctrrrs rrr,rrzO Dinat encoder's outputs, with potential hidden states and attentions. ) custom_introceZdZUdZdZej dzed<dZe ej dfdzed<dZ e ej dfdzed<dZ e ej dfdzed<y)DinatEncoderOutputa reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nlast_hidden_state. hidden_states attentionsreshaped_hidden_states) __name__ __module__ __qualname____doc__r"torch FloatTensor__annotations__r#tupler$r%rrr!r!0s}37u((4/6:>M5**C/047>7;Je'',-4;CGE%"3"3S"89D@Grr!zW Dinat model's outputs that also contains a pooling of the last hidden states. ceZdZUdZdZej dzed<dZej dzed<dZ e ej dfdzed<dZ e ej dfdzed<dZ e ej dfdzed<y) DinatModelOutputa pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`, *optional*, returned when `add_pooling_layer=True` is passed): Average pooling of the last layer hidden-state. reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nr" pooler_output.r#r$r%) r&r'r(r)r"r*r+r,r1r#r-r$r%r.rrr0r0Fs 37u((4/6.2M5$$t+2:>M5**C/047>7;Je'',-4;CGE%"3"3S"89D@Grr0z1 Dinat outputs for image classification. ceZdZUdZdZej dzed<dZej dzed<dZ e ej dfdzed<dZ e ej dfdzed<dZ e ej dfdzed<y) DinatImageClassifierOutputa7 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Classification (or regression if config.num_labels==1) loss. logits (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`): Classification (or regression if config.num_labels==1) scores (before SoftMax). reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nlosslogits.r#r$r%) r&r'r(r)r4r*r+r,r5r#r-r$r%r.rrr3r3_s &*D%  d ")'+FE   $+:>M5**C/047>7;Je'',-4;CGE%"3"3S"89D@Grr3cfeZdZdZfdZdej dzdeejfdZ xZ S)DinatEmbeddingsz6 Construct the patch and position embeddings. ct|t||_t j |j |_t j|j|_ yr) super__init__DinatPatchEmbeddingspatch_embeddingsr LayerNorm embed_dimnormDropouthidden_dropout_probdropoutselfconfig __class__s rr:zDinatEmbeddings.__init__sG  4V <LL!1!12 zz&"<"<= r pixel_valuesNreturncl|j|}|j|}|j|}|Sr)r<r?rB)rDrG embeddingss rforwardzDinatEmbeddings.forwards4**<8 YYz* \\*- r) r&r'r(r)r:r*r+r-TensorrK __classcell__rFs@rr7r7zs4>E$5$5$<u||ATrr7c`eZdZdZfdZdej dzdejfdZxZ S)r;z This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, height, width, hidden_size)` to be consumed by a Transformer. c Pt||j}|j|j}}||_|dk(rn t dt jt j|j|dzdddt j|dz|ddd|_ y)Nz2Dinat only supports patch size of 4 at the moment.rrrRrRrr) kernel_sizestridepadding) r9r: patch_size num_channelsr> ValueErrorr SequentialConv2d projection)rDrErYrZ hidden_sizerFs rr:zDinatPatchEmbeddings.__init__s && $*$7$79I9Ik ( ? QR R-- IId'')9vV\fl m IIkQ& PV`f g rrGNrHc|j\}}}}||jk7r td|j|}|j dddd}|S)NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rrRrr)shaperZr[r^permute)rDrG_rZheightwidthrJs rrKzDinatPatchEmbeddings.forwards`)5););&< 4,, ,w __\2 ''1a3 r) r&r'r(r)r:r*r+rLrKrMrNs@rr;r;s/  " E$5$5$<  rr;ceZdZdZej fdedejddffd Zde jde jfdZ xZ S) DinatDownsamplerz Convolutional Downsampling Layer. Args: dim (`int`): Number of input channels. norm_layer (`nn.Module`, *optional*, defaults to `nn.LayerNorm`): Normalization layer class. dim norm_layerrHNct|||_tj|d|zdddd|_|d|z|_y)NrRrSrTrUF)rVrWrXbias)r9r:rhrr] reductionr?)rDrhrirFs rr:zDinatDownsampler.__init__sE 3CVF\binoq3w' r input_featurec|j|jddddjdddd}|j|}|S)NrrrrR)rlrbr?)rDrms rrKzDinatDownsampler.forwardsJ}'<'