^jUPJdZddlmZddlmZddlmZmZmZm Z m Z m Z m Z ddl Z ddlmZddlmZmZddlmZmZmZmZmZmZdd lmZdd lmZdd lmZdd l m!Z!m"Z"dd l#m$Z$m%Z%dgZ&eeejNZ(GddejRZ* d;de+de+de,de,de+de,dee e,e+e,e+ffdZ- dd$e1de0fd1Z5e%d>d$e1de0fd2Z6e%d>d$e1de0fd3Z7e%d>d$e1de0fd4Z8e%d>d$e1de0fd5Z9e%d>d$e1de0fd6Z:e%d>d$e1de0fd7Z;e%d>d$e1de0fd8Zd$e1de0fd9Z=e%d>d$e1de0fd:Z>y)?a ReXNet A PyTorch impl of `ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network` - https://arxiv.org/abs/2007.00992 Adapted from original impl at https://github.com/clovaai/rexnet Copyright (c) 2020-present NAVER Corp. MIT license Changes for timm, feature extraction, and rounded channel variant hacked together by Ross Wightman Copyright 2020 Ross Wightman )partial)ceil)AnyDictListOptionalTupleTypeUnionNIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)ClassifierHeadcreate_act_layer ConvNormActDropPathmake_divisibleSEModule)build_model_with_cfg)efficientnet_init_weights)feature_take_indices) checkpointcheckpoint_seq)generate_default_cfgsregister_modelRexNet) norm_layerceZdZdZ ddedededeeefdededed ed ed ee jffd Z dd e defdZ dejdejfdZxZS)LinearBottleneckzLinear bottleneck block for ReXNet. A mobile inverted residual bottleneck block as used in MobileNetV2 and subsequent models. in_chsout_chsstridedilation exp_ratiose_ratioch_div act_layer dw_act_layer drop_pathc | | d} t||dk(xr|d|dk(xr||k|_||_||_|dk7r.t t ||z|}t||fd|i| |_n |}d|_t||fd||d|d d | |_ |dkDr*t|fd t t||z|i| |_ nd|_ t| |_t||dfd d i| |_| |_y) aInitialize LinearBottleneck. Args: in_chs: Number of input channels. out_chs: Number of output channels. stride: Stride for depthwise conv. dilation: Dilation rates. exp_ratio: Expansion ratio. se_ratio: Squeeze-excitation ratio. ch_div: Channel divisor. act_layer: Activation layer for expansion. dw_act_layer: Activation layer for depthwise. drop_path: Drop path module. devicedtyperr?divisorr(NF) kernel_sizer#r$groups apply_act rd_channelsr5)super__init__ use_shortcut in_channels out_channelsrroundrconv_expconv_dw SEWithNormintseract_dwconv_pwlr*)selfr!r"r#r$r%r&r'r(r)r*r-r.dddw_chs __class__s ]/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/models/rexnet.pyr8zLinearBottleneck.__init__(s1: / "aK\HQK8A;,F\6U\K\!# ?#E&9*<$=vNF'R)RrRDMF DM"    a[      a< j^CQYHYDZ\b5cjgijDGDG&|4 #FGQN%N2N "expreturncJ|r|jjS|jS)zGet feature channel count. Args: exp: Return expanded channels if True. Returns: Number of feature channels. )r>r;)rDrJs rH feat_channelszLinearBottleneck.feat_channelses!-0t||((FT5F5FFrIxc|}|j|j|}|j|}|j|j|}|j|}|j |}|j r_|j |j |}tj|ddd|jf|z|dd|jdfgd}|S)zoForward pass. Args: x: Input tensor. Returns: Output tensor. Nrr)dim) r=r>rArBrCr9r*torchcatr:)rDrNshortcuts rHforwardzLinearBottleneck.forwardps == $ a A LLO 77  A KKN MM!    ~~)NN1% 1Q$"2"2 223h>!TEUEUEVBV@WX^_`ArI) )rrr/rswishrelu6NNNF)__name__ __module__ __qualname____doc__r@r floatstrrnnModuler8boolrMrQTensorrT __classcell__rGs@rHr r "s)/" $ '-1;#;#;# ;# CHo ;#  ;#;#;#;#;#  *;#z G G# G%,,rIr r/ width_mult depth_mult initial_chs final_chsr&r'rKc Dgd}gd}|Dcgc]}t||z}}tt|D cgc]\} }|gdg|| dz zzc}} g}dg|dzdgt|ddzz} t|dddz} |dkr||z n|} g} t| dzD]8}| j t t | |z| | || dzdzz z } :d g|d|dzz|gt|d dzz}tt| | ||Scc}wcc}} w) asGenerate ReXNet block configuration. Args: width_mult: Width multiplier. depth_mult: Depth multiplier. initial_chs: Initial channel count. final_chs: Final channel count. se_ratio: Squeeze-excitation ratio. ch_div: Channel divisor. Returns: List of tuples (out_channels, exp_ratio, stride, se_ratio). )rrjr2r2)rrjrjrjrrjrrNr2r/r0rUrj) rsum enumeraterangeappendrr<listzip)rerfrgrhr&r'layersstrideselementidx exp_ratiosdepthbase_chs out_chs_listi se_ratioss rH _block_cfgr}sT* F G8> ?Wd7Z'( ?F ?9U\K]^<3G9sfSkAo66^`bcGvayA3VABZ#88J q NQ E+5+;{Z'HL 5A: 3N5J1F+GQWXYI!c!1223q F1I-.(c&*o1MMI L*gyA BB@^s DD block_cfgprev_chs output_strider(r)drop_path_ratec J|| d} |g} g} d} d}g}t|}t|D]\}\}}}}|}|dkDr0|dk(rdnd|dz }| t| d| |gz } | |k\r||z}d}||z|dz z }|d kDr t|nd }|j t d||||||f|||||d | | |z} |}|}| |dj gz } td |z| }| t| d| dt|dz gz } |j t||fd|i| || fS)aBuild ReXNet blocks from configuration. Args: block_cfg: Block configuration list. prev_chs: Previous channel count. width_mult: Width multiplier. ch_div: Channel divisor. output_stride: Target output stride. act_layer: Activation layer name. dw_act_layer: Depthwise activation layer name. drop_path_rate: Drop path rate. Returns: Tuple of (features list, feature_info list). r,rjrrstemz features.)num_chs reductionmodulerUN) r!r"r%r#r$r&r'r(r)r*ir0r() lenrndictrrpr rMrr)r~rrer'rr(r)rr-r.rEfeat_chs feature_info curr_strider$features num_blocks block_idxchsr%r#r& next_dilationfname block_dprr*pen_chss rH _build_blocksrs6U +BzHLKHHYJ9B99M35 5CFH A:'1nFIi!m_2ME T(2,+V[\] ]Lm+ (6 1 "Y.*q.A +4r>HY't (   .%     v  Xb\//122536TJ.?GT(2,+PYZ]^fZgjkZkYlNmnooL OOK'MYM"MN \ !!rIceZdZdZ d'dedededededed ed ed ed ed edededeffd Zejjd(de de ee ffdZejjd)de ddfdZejjdej"fdZd*dedeeddfdZ d+dej*deeeeefde de dede deeej*eej*eej*fffdZ d,deeeefd e d!e deefd"Zdej*dej*fd#Zd(dej*d$e dej*fd%Zdej*dej*fd&ZxZS)-rzReXNet model architecture. Based on `ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network` - https://arxiv.org/abs/2007.00992 Nin_chans num_classes global_poolrrgrhrerfr&r'r(r) drop_raterc t|||d}||_||_| |_d|_|dvsJ|dkrd|z nd}t t||z| }t||dfd| d ||_ t||||| | }t|||| || | |fi|\}|_ |d jx|_|_t!j"||_t'|j||| fi||_t+|y ) aInitialize ReXNet. Args: in_chans: Number of input channels. num_classes: Number of classes for classification. global_pool: Global pooling type. output_stride: Output stride. initial_chs: Initial channel count. final_chs: Final channel count. width_mult: Width multiplier. depth_mult: Depth multiplier. se_ratio: Squeeze-excitation ratio. ch_div: Channel divisor. act_layer: Activation layer name. dw_act_layer: Depthwise activation layer name. drop_rate: Dropout rate. drop_path_rate: Drop path rate. r,F) r/rr0r2rj)r#r(rN)r7r8rrrgrad_checkpointingrr<rrr}rrr; num_featureshead_hidden_sizer_ Sequentialrrheadr)rDrrrrrgrhrerfr&r'r(r)rrr-r.rE stem_base_chsstem_chsr~rrGs rHr8zRexNet.__init__s(J  /&  ""' ++++5+;Z !% (B"CVT(A[a9[XZ[ z:{IxY_` &3         '  ' #$#5=RL4M4MMD1 x0 "4#4#4k;PY`]_` !$'rIcoarserKc tdd}|S)zGroup matcher for parameter groups. Args: coarse: Whether to use coarse grouping. Returns: Dictionary of grouped parameters. z^stemz^features\.(\d+))rblocks)r)rDrmatchers rH group_matcherzRexNet.group_matcher>s& rIenablec||_y)zEnable or disable gradient checkpointing. Args: enable: Whether to enable gradient checkpointing. N)r)rDrs rHset_grad_checkpointingzRexNet.set_grad_checkpointingNs #)rIc.|jjS)zTGet the classifier module. Returns: Classifier module. )rfc)rDs rHget_classifierzRexNet.get_classifierWsyy||rIc||_||R||d}||n|jjj}t |j |||j fi||_y|jj||y)zReset the classifier. Args: num_classes: Number of classes for new classifier. global_pool: Global pooling type. Nr,)rrr pool_typerrrreset)rDrrr-r.rErs rHreset_classifierzRexNet.reset_classifier`st'  !2"U3B'2'> DIIDYDYDcDcI&t'8'8+yRVR`R`gdfgDI IIOOK 5rIrNindicesnorm stop_early output_fmtintermediates_onlyc||dvsJdg}|jDcgc]"}t|djdd$} }tt | |\} } | D cgc]} | |  } } | | } |j |}t jjs|s |j} n|jd| dz} t| D]Z\}}|jr+t jjs t||}n||}|| vsJ|j|\|r|S||fScc}wcc} w)a Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features Returns: )NCHWzOutput shape must be NCHW.r.rNr)rr@splitrrrrQjit is_scriptingrrnrrrp)rDrNrrrrr intermediatesinfo stage_ends take_indices max_indexr{stagesfeat_idxstages rHforward_intermediateszRexNet.forward_intermediatesos=*Y&D(DD& EIEVEVWTc$x...s3B78W W"6s:"P i/;IM2F(0 (OHe&&uyy/E/E/Gua(!H<'$$Q'  (  -/X*  c 0%,,5<< Sell S S S  %,, rIvariant pretrainedc Btd}tt||fd|i|S)zCreate a ReXNet model. Args: variant: Model variant name. pretrained: Load pretrained weights. **kwargs: Additional model arguments. Returns: ReXNet model instance. T)flatten_sequential feature_cfg)rrr)rrkwargsrs rH_create_rexnetrs8$/K       rIurlc 2|dddddttdddd |S) zCreate default configuration dictionary. Args: url: Model weight URL. **kwargs: Additional configuration options. Returns: Configuration dictionary. r)r2r)rg?bicubicz stem.convzhead.fcmit) rr input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierlicenser )rrs rH_cfgrs54}SYI%.B!   #  rIztimm/) hf_hub_idgffffff?)r2 rz apache-2.0)rr test_crop_pcttest_input_sizeri-.)rrrrrr) zrexnet_100.nav_in1kzrexnet_130.nav_in1kzrexnet_150.nav_in1kzrexnet_200.nav_in1kzrexnet_300.nav_in1kzrexnetr_100.untrainedzrexnetr_130.untrainedzrexnetr_150.untrainedzrexnetr_200.sw_in12k_ft_in1kzrexnetr_300.sw_in12k_ft_in1kzrexnetr_200.sw_in12kzrexnetr_300.sw_in12kc td|fi|S)zReXNet V1 1.0x rexnet_100rrrs rHrr#s , =f ==rIc td|fddi|S)zReXNet V1 1.3x rexnet_130re?rrs rHrr) , Ms Mf MMrIc td|fddi|S)zReXNet V1 1.5x rexnet_150re?rrs rHrr/rrIc td|fddi|S)zReXNet V1 2.0x rexnet_200re@rrs rHrr5rrIc td|fddi|S)zReXNet V1 3.0x rexnet_300re@rrs rHrr;rrIc td|fddi|S)z*ReXNet V1 1.0x w/ rounded (mod 8) channels rexnetr_100r'rrrs rHrrAs - HA H HHrIc "td|fddd|S)z*ReXNet V1 1.3x w/ rounded (mod 8) channels rexnetr_130rrrer'rrs rHr r G - XA XQW XXrIc "td|fddd|S)z*ReXNet V1 1.5x w/ rounded (mod 8) channels rexnetr_150rrr rrs rHr r Mr rIc "td|fddd|S)z*ReXNet V1 2.0x w/ rounded (mod 8) channels rexnetr_200rrr rrs rHrrSr rIc "td|fddd|S)z+ReXNet V1 3.0x w/ rounded (mod 16) channels rexnetr_300rrr rrs rHrrYs - YB YRX YYrI)r/r/rrrUr)rrrVrWrUNN)rrX)?r\ functoolsrmathrtypingrrrrr r r rQtorch.nnr_ timm.datar r timm.layersrrrrrr_builderr_efficientnet_builderr _featuresr _manipulaterr _registryrr__all__ BatchNorm2dr?r`r r]r@r}r^rrrarr default_cfgsrrrrrrr r rrrrIrHr s @@@ Aii*<+3< *X".. 9 cryycN  %C%C%C%C %C  %C  %C %UC& '(%CX # "@"c5#u456@"@"@" @"  @"  @"@"@" 4 ?Dc3h0 01@"FkRYYk\CT*cT#s(^&%'2'2'2'2'2!V!V!V$(S-Q]%_%)S-Q]%_!S-Q]_!S-Q]_'& 4>4>f>> N4NfNN N4NfNN N4NfNN N4NfNN IDIvII YDYvYY YDYvYY YDYvYY ZDZvZZrI