^jdZddlmZmZmZmZmZddlZddlmZddl m Z ddl m Z ddl mZd ZGd d ej ZGd d ej Zy)z Selective Kernel Convolution/Attention Paper: Selective Kernel Networks (https://arxiv.org/abs/1903.06586) Hacked together by / Copyright 2020 Ross Wightman )ListOptionalTupleTypeUnionN)nn) ConvNormAct)make_divisible)_assertcnt|ttfr|D]}t|cS|dk\r|dzsJy)N) isinstancelisttuple _kernel_valid)kkis g/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/layers/selective_kernel.pyrrs?!dE]# %B $ $ % 6a!e ec eZdZddejej ddfdedededeejdeejf fd Z d Z xZ S) SelectiveKernelAttnr Nchannels num_paths attn_channels act_layer norm_layerc||d}t |||_tj||fddd||_||fi||_|d|_tj|||zfddd||_y)z| Selective Kernel Attention Module Selective Kernel attention mechanism factored out into its own module. devicedtyper F) kernel_sizebiasT)inplaceN) super__init__rrConv2d fc_reducebnact fc_select) selfrrrrrr"r#dd __class__s rr(zSelectiveKernelAttn.__init__s / "8]\PU\Y[\]1b1T*=(Y2FhTU\aheghrct|jd|jk(d|jdj dd}|j |}|j |}|j|}|j|}|j\}}}}|j||j||jz||}tj|d}|S)Nr )rrT)keepdimdim) r shapersummeanr*r+r,r-viewtorchsoftmax)r.xBCHWs rforwardzSelectiveKernelAttn.forward0s dnn,b1 EE!HMM&$M / NN1  GGAJ HHQK NN1 WW 1a FF1dnna4>>&91a @ MM! #r) __name__ __module__ __qualname__rReLU BatchNorm2dintrModuler(rA __classcell__r0s@rrrso!#)+*,..iii i BII i RYY i. rrc!BeZdZddddddddddejej ddddfdedeedeeee efd ed ed ed e d eedede de de ejde ejdee ejdee ejffd ZdZxZS)SelectiveKernelNr g?T in_channels out_channelsr$stridedilationgroupsrd_ratio rd_channels rd_divisorkeep_3x3 split_inputrraa_layer drop_layerc ||d}t||xs|}|xsddg}t|t|ts|gdz}| r(|Dcgc] }||dz zdz}}dgt |z}n|gt |z}t ||_||_||_| |_ |jr#||j zdk(sJ||j z}t||}td ||| | ||d|}tjt||Dcgc]\}}t||f||d|c}}|_|xst#||z| }t%||j |fi||_y cc}wcc}}w) a: Selective Kernel Convolution Module As described in Selective Kernel Networks (https://arxiv.org/abs/1903.06586) with some modifications. Largest change is the input split, which divides the input channels across each convolution path, this can be viewed as a grouping of sorts, but the output channel counts expand to the module level value. This keeps the parameter count from ballooning when the convolutions themselves don't have groups, but still provides a noteworthy increase in performance over similar param count models without this attention layer. -Ross W Args: in_channels: module input (feature) channel count out_channels: module output (feature) channel count kernel_size: kernel size for each convolution branch stride: stride for convolutions dilation: dilation for module as a whole, impacts dilation of each branch groups: number of groups for each branch rd_ratio: reduction factor for attention features keep_3x3: keep all branch convolution kernels as 3x3, changing larger kernels for dilations split_input: split input channels evenly across each convolution branch, keeps param count lower, can be viewed as grouping by path, output expands to module out_channels count act_layer: activation layer to use norm_layer: batchnorm/norm layer to use aa_layer: anti-aliasing module drop_layer: spatial drop module in convs (drop block, etc) r!rrr r)rPrRrrrXrY)r$rQ)divisorN)r'r(rrrlenrrNrOrWmindictr ModuleListzipr pathsr rattn)r.rNrOr$rPrQrRrSrTrUrVrWrrrXrYr"r#r/r conv_kwargsdrr0s rr(zSelectiveKernel.__init__?sZ / #2{ !+aV k"+t,&-!+K 9DEAAE*a/EHE#K 00K zC $44H[)&(&   /14 44%7K\6*<&I**<8:< ]]K2$41  \ \q1 \P[ \$45 $b~lX6MWa'b ' dnnmZWYZ +F $4s E& E+ c|jr_tj||j|jzd}t |j Dcgc]\}}|||}}}n|j Dcgc] }|| }}tj|d}|j|}||z}tj|d}|Scc}}wcc}w)Nr r4) rWr:splitrNr enumeratercstackrdr7)r.r<x_splitiopx_pathsx_attns rrAzSelectiveKernel.forwards   kk!T%5%5%GKG3rwsI 65 $# ""))"JXbiiXr