^j"<dZddlmZmZddlZddlmZddlmcmZ  ddejde de de de d e d e d e fd ZGd dejZdde de d e fdZGddejZ dde dee ee fde deee eee ffdZy)a> DropBlock, DropPath PyTorch implementations of DropBlock and DropPath (Stochastic Depth) regularization layers. Papers: DropBlock: A regularization method for convolutional networks (https://arxiv.org/abs/1810.12890) Deep Networks with Stochastic Depth (https://arxiv.org/abs/1603.09382) Code: DropBlock impl inspired by two Tensorflow impl that I liked: - https://github.com/tensorflow/tpu/blob/master/models/official/resnet/resnet_model.py#L74 - https://github.com/clovaai/assembled-cnn/blob/master/nets/blocks.py Hacked together by / Copyright 2020 Ross Wightman )ListUnionNx drop_prob block_size gamma_scale with_noiseinplacecouple_channels scale_by_keepc |j\}} } } t|| t|| } } t||z| z| zt| | zz t| | z dz| | z dzzz }||rdn| | | f}tj5tj ||j |jj|}tj|| | fd| dz| dzf}| dzdk(s| dzdk(r|d| dzdzd| dzdzdf}d|z }ddd|rtj5tjj}|jddd|r"|jj|S|zz}|S|rtj5j|j!tj" j%j'd z }|j|j!|j ddd|r|j|S|z}|S#1swY>xYw#1swYxYw#1swY>xYw) a DropBlock. See https://arxiv.org/pdf/1810.12890.pdf DropBlock with an experimental gaussian noise option. Args: x: Input tensor of shape (B, C, H, W). drop_prob: Probability of dropping a block. block_size: Size of the block to drop. gamma_scale: Scale factor for the drop probability. with_noise: If True, add gaussian noise to dropped regions instead of zeros. inplace: If True, perform operation in-place. couple_channels: If True, all channels share the same drop mask (per the original paper). If False, each channel gets an independent mask. scale_by_keep: If True, scale kept activations to maintain expected values. Returns: Tensor with dropped blocks, same shape as input. )dtypedevice) kernel_sizestridepaddingr.N?)rgHz>)shapeminfloattorchno_grademptyrr bernoulli_F max_pool2d empty_likenormal_mul_add_numeltofloat32sumadd)rrrrr r r r BCHWkhkwgamma noise_shape block_mask keep_masknoisenormalize_scales [/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/layers/drop.py drop_block_2dr5sb8JAq!Q Q Z!3B + )A-1 2U27^ CeQQSVVWZ\]`b\bef\fLgFh hE?a1a8K $[[AGGAHHMXXY^_ \\ R1WbAg&   6Q;"q&A+#C"q&Aa1 $EFJO $" ]]_ #$$Y/779E JJz " #  FF9  " "5 ) HI %A H  <"+//"3illl6W6[6[6]6a6abf6g"g11!'':; <  FF9  HI A HM$$$ # # < __class__s r4r=zDropBlock2d.__init__ts "&$$ .*'/ VA' OPQsRSTU Vc |jr |js|St||j|j|j|j |j |j|jS)N)rrrr r r r ) trainingrr5rrr r r r r@rs r4forwardzDropBlock2d.forwards[}}DNNH nn((LL 00,,  rEg?rFFTT) __name__ __module__ __qualname____doc__rintboolr=rI __classcell__rDs@r4r7r7fsu  #!$$!$("&VVV V  V  V"V V6  rEr7rGc|dk(s|s|Sd|z }|jdfd|jdz zz}|j|j|}|dkDr|r|j |||zS)a(Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rr)r)rndim new_emptyrdiv_)rrrGr keep_probr random_tensors r4 drop_pathr[swBhI I WWQZMDAFFQJ/ /EKK&11)t|||_||_yN)r<r=rr )r@rr rDs r4r=zDropPath.__init__s "*rEcZt||j|j|jSr_)r[rrGr rHs r4rIzDropPath.forwards!DNNDMM4;M;MNNrEc6dt|jddS)Nz drop_prob=z0.3f)roundr)r@s r4 extra_reprzDropPath.extra_reprsE$..3D9::rE)rUT) rLrMrNrOrrQr=rIrdrRrSs@r4r]r]s&+%+T+ O;rEr]drop_path_ratedepths stagewisereturnct|trC|r tdtjd||dDcgc]}|j }}|St |}|rEtjd||dj|Dcgc]}|j}}|Stjd||dDcgc]}|j }}|Scc}wcc}wcc}w)a Generate drop path rates for stochastic depth. This function handles two common patterns for drop path rate scheduling: 1. Per-block: Linear increase from 0 to drop_path_rate across all blocks 2. Stage-wise: Linear increase across stages, with same rate within each stage Args: drop_path_rate: Maximum drop path rate (at the end). depths: Either a single int for total depth (per-block mode) or list of ints for depths per stage (stage-wise mode). stagewise: If True, use stage-wise pattern. If False, use per-block pattern. When depths is a list, stagewise defaults to True. Returns: For per-block mode: List of drop rates, one per block. For stage-wise mode: List of lists, drop rates per stage. z;stagewise=True requires depths to be a list of stage depthsrcpu)r) isinstancerP ValueErrorrlinspaceitemr&splittolist)rerfrgrdpr total_depths r4calculate_drop_path_ratesrss,&# Z[ [!&>6RW!XYAqvvxYY &k ',~~a]b'c'i'ijp'qr!188:rCrJ&+^^A~{[`%ab1668bCbJZscsC C!C&rJ)rUFT)F)rOtypingrrrtorch.nnnntorch.nn.functional functionalrTensorrrPrQr5Moduler7r[r]rsrEr4r|s      $"K <<K K K  K  K  K K K \5 "))5 pE$t( ;ryy ;$ &&c49n%&& 4;T%[) )* &rE