^j+& dZddlZddlmZmZddlmZddlmZm Z ddl m Z m Z m Z ddlmZdd lmZmZdd lmZGd d ej*ZGd dej*ZddZddZeededeededdZeddefdZeddefdZeddefdZeddefdZeddefdZy)a Selective Kernel Networks (ResNet base) Paper: Selective Kernel Networks (https://arxiv.org/abs/1903.06586) This was inspired by reading 'Compounding the Performance Improvements...' (https://arxiv.org/abs/2001.06268) and a streamlined impl at https://github.com/clovaai/assembled-cnn but I ended up building something closer to the original paper with some modifications of my own to better balance param count vs accuracy. Hacked together by / Copyright 2020 Ross Wightman N)OptionalType)nnIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)SelectiveKernel ConvNormAct create_attn)build_model_with_cfg)register_modelgenerate_default_cfgs)ResNetc"eZdZdZddddddddej ej ddddddfdedededeejded ed ee d ed ed eede ejde ejdee ejdee ejdeejdeejf fd Z dZ dZxZS)SelectiveKernelBasicr N@inplanesplanesstride downsample cardinality base_width sk_kwargs reduce_firstdilationfirst_dilation act_layer norm_layer attn_layeraa_layer drop_block drop_pathcr||d}t||xsi}td | | d|}|dk(sJd|dk(sJd||z}||jz}| xs| } t ||f|| ||d|||_t ||fd| d d ||_t| |fi||_ | d |_ ||_ ||_ y)Ndevicedtyperrr z)BasicBlock only supports cardinality of 1rz0BasicBlock doest not support changing base width)rrr! drop_layerF) kernel_sizer apply_actTinplace) super__init__dict expansionr conv1r conv2r seactrr#)selfrrrrrrrrrrrrr r!r"r#r&r'dd conv_kwargs first_planes outplanes __class__s \/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/models/sknet.pyr1zSelectiveKernelBasic.__init__s* / O LY:LL aL!LLRS!SS- T^^+ '38$    #!       !        j):r:T*$"ct|jjdd>tjj |jjj yyNweight)getattrr5bnrinitzeros_rBr8s r>zero_init_lastz#SelectiveKernelBasic.zero_init_lastQ< 4::==(D 1 = GGNN4::==// 0 >r?c(|}|j|}|j|}|j|j|}|j|j|}|j|j |}||z }|j |}|SN)r4r5r6r#rr7r8xshortcuts r>forwardzSelectiveKernelBasic.forwardUs JJqM JJqM 77  A >> %q!A ?? &x0H X  HHQKr?__name__ __module__ __qualname__r3rReLU BatchNorm2dintrModuler2rr1rHrO __classcell__r=s@r>rrs=I .2  (, !,0)+*,..4826.2-1'5#5#5# 5# !+ 5#  5#5# ~5#5#5#%SM5#BII5#RYY5#!bii15#tBII/5# !+!5#"  *#5#n1 r?rc"eZdZdZddddddddej ej ddddddfdedededeejd ed ed ee d ed edeede ejde ejdee ejdee ejdeejdeejf fd Z dZ dZxZS)SelectiveKernelBottleneckr Nrrrrrrrrrrrrrr r!r"r#c ||d}t||xsi}td | | d|}tt j ||dz z|z}||z}||j z}| xs| } t||fddi||_t||f|| |||d|||_ t||fddd||_ t| |fi||_ | d |_||_||_y) Nr%r(rr+r )rrgroupsr!r)F)r+r,Tr-r/)r0r1r2rVmathfloorr3r r4r r5conv3r r6r7rr#)r8rrrrrrrrrrrrr r!r"r#r&r'r9r:widthr;r<r=s r>r1z"SelectiveKernelBottleneck.__init__gs* / O LY:LL DJJvb9:[HI , T^^+ '38 <VQV+V $    #!       ! aqEaU`a j):r:T*$"r?ct|jjdd>tjj |jjj yyrA)rCrarDrrErFrBrGs r>rHz(SelectiveKernelBottleneck.zero_init_lastrIr?cJ|}|j|}|j|}|j|}|j|j|}|j|j |}|j |j |}||z }|j |}|SrK)r4r5rar6r#rr7rLs r>rOz!SelectiveKernelBottleneck.forwards JJqM JJqM JJqM 77  A >> %q!A ?? &x0H X  HHQKr?rPrYs@r>r[r[ds=I .2  (, !,0)+*,..4826.2-1'/#/#/# /# !+ /#  /#/# ~/#/#/#%SM/#BII/#RYY/#!bii1/#tBII//# !+!/#"  *#/#b1 r?r[c &tt||fi|SrK)r r)variant pretrainedkwargss r>_create_skresnetris"     r?c 2|dddddttdddd |S) Ni)r*rk)rlg?bicubicr4fcz apache-2.0) url num_classes input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierlicenser)rorhs r>_cfgrzs3=vI%.BT    r?ztimm/) hf_hub_idzconv1.0)rw)zskresnet18.ra_in1kzskresnet34.ra_in1kzskresnet50.untrainedzskresnet50d.untrainedzskresnext50_32x4d.ra_in1kreturnc tddd}ttgdt|d}td |fit|fi|S) zConstructs a Selective Kernel ResNet-18 model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. ?Trd_ratio rd_divisor split_input)rrrrFblocklayers block_argsrH skresnet18r2rrirgrhr model_argss r>rrJeEI"rrrr?c td}ttgdt|d}td|fit|fi|S)zConstructs a Select Kernel ResNet-50 model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. TrrrFr skresnet50r2r[rirs r>rrsF&I' XaIbJ L* SZ8R68R SSr?c td}ttgddddt|d}td |fit|fi|S) zConstructs a Select Kernel ResNet-50-D model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. Trr deeprF)rr stem_width stem_typeavg_downrrH skresnet50drrs r>rrsN&I' W]hl),UDJ M: Tj9SF9S TTr?c tddd}ttgdddt|d}td |fit|fi|S) zConstructs a Select Kernel ResNeXt50-32x4d model. This should be equivalent to the SKNet-50 model in the Select Kernel Paper g?rFrrr\r)rrrrrrHskresnext50_32x4drrs r>rrsQ druEI' "YZ),UDJ / ZtJ?YRX?Y ZZr?)F)) __doc__r_typingrrtorchr timm.datarr timm.layersr r r _builderr _registryrrresnetrrWrr[rirz default_cfgsrrrrrr/r?r>rs&  !AAA*<J299JZE EP %11 F!!%!8 &  Tf T T Tf T T Tf T T Uv U U[V[[r?