^j]dZddlmZmZddlZddlZddlZddlmZddl m Z ddlm Z m Z m Z ddlmZddlmZmZmZmZmZmZdd ldd lmZgd Zej6eZd Zd ZdZ e!ee Z"e e e e#e fZ$dZ%dZ&ddZ'ddZ(dZ)dZ*dZ+d dZ, d!dZ-GddZ.d"dZ/d#dej`fdZ1y)$z EfficientNet, MobileNetV3, etc Builder Assembles EfficieNet and related network feature blocks from string definitions. Handles stride, dilation calculations, and selects feature extraction points. Hacked together by / Copyright 2019, Ross Wightman )CallableOptionalN)deepcopy)partial)AnyDictList) CondConv2dget_condconv_initializer get_act_layerget_attnmake_divisible LayerType)*) named_modules) EfficientNetBuilder BlockArgsdecode_arch_defefficientnet_init_weightsresolve_bn_argsresolve_act_layerround_channelsBN_MOMENTUM_TF_DEFAULTBN_EPS_TF_DEFAULTFgGz?gMbP?)momentumepsc*tjSN) _BN_ARGS_TFcopyl/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/models/_efficientnet_builder.pyget_bn_args_tfr%+s    r#cni}|jdd}|||d<|jdd}|||d<|S)N bn_momentumrbn_epsr)pop)kwargsbn_argsr'r(s r$rr/sKG**]D1K)  ZZ$ 'F  Nr#c8t|jd|S)N act_layer)r r))r*defaults r$rr:s K9 ::r#c.|s|St||z|||S)z2Round number of filters based on depth multiplier.) round_limit)r)channels multiplierdivisor channel_minr0s r$rr>s!  (Z/+S^ __r#c4|rtj|yyr)_loggerinfo)msg conditions r$ _log_info_ifr:Es Sr#c|jr t|S|jdDcgc] }t|c}Scc}w)N.)isdigitintsplit)ssks r$ _parse_ksizerBJs4 zz|2w " .1A...sAct|tsJ|jd}|d}|dd}i}d}|D]}|dk(rd} |dk(rd}|jd rf|d}|dd}|d k(r t d }nE|d k(r t d }n4|dk(r t d}n#|dk(r t d}n|dk(r t d}n|||<t jd|} t | dk\s| dd\}}|||<d |vr|d nd} d|vrt|dnd} d|vrt|dnd} d|vrt|dnd} t|d}t|t|dt|d| }|dk(r|jtt|d| | t|dt|jd d!|dut|jd"ddkD#d$|vrpt|d$|d%<n]|d&k(s|d'k(ro|jtt|d| t|jd d!|d'k(|d'k(xs|dut|jd"ddkD(n|d)k(rU|jtt|d| t|d| t|jd d!|du*n|d+k(r,|jtt|d|du,nY|d-k(r|d|vrt|dnd} d|vrt|dnd} |jt| t|d| t|dt|jd d!|du.n|d/k(rbt|d"}|jtt|dt|d0||t|jd1d|du2nq|d3k(rbt|d"}|jtt|dt|d0||t|jd1d|du2n Jd4|zd5|vrt|d5|d6<||fS)7a@ Decode block definition string Gets a list of block arg (dicts) through a string notation of arguments. E.g. ir_r2_k3_s2_e1_i32_o16_se0.25_noskip All args can exist in any order with the exception of the leading string which is assumed to indicate the block type. leading string - block type ( ir = InvertedResidual, ds = DepthwiseSep, dsa = DeptwhiseSep with pw act, cn = ConvBnAct) r - number of repeat blocks, k - kernel size, s - strides (1-9), e - expansion ratio, c - output channels, se - squeeze/excitation ratio n - activation fn ('re', 'r6', 'hs', or 'sw') Args: block_str: a string representation of block arguments. Returns: A list of block args (dicts) Raises: ValueError: if the string def not properly specified (TODO) _rrNnoskipFskipTnrerelur6relu6hs hard_swishswswishmimishz(\d.*)apfcrcs) block_typeout_chsstrider-irrAesed)dw_kernel_sizeexp_kernel_sizepw_kernel_size exp_ratiose_ratiorEs2dcc num_expertsdsdsa)rarcrepw_actrErfer)rbrcrd force_in_chsrerEcn) kernel_sizerFuir)dw_kernel_size_startdw_kernel_size_middw_kernel_size_endrdrerEmhahv)ra num_headskey_dim value_dim kv_striderEmqazUnknown block type (%s)gs group_size) isinstancestrr? startswithr rHlenrBr>dictupdatefloatget) block_stropsrYoptionsrFopkeyrvvaluesplitsr-start_kernel_sizeend_kernel_sizerm num_repeat block_argskv_dims r$_decode_block_strrQs2 i %% % //# CQJ ab'CG D% >D 6\D ]]3 Q%C12ADy%f-d%g.d%l3d%g.d%f- GCLXXi,F6{a#BQZ U$ 9%>!$w DI69Wn WS\2!477Nl73<0O)-3wt}%aLWS\"JGCL!73<  J T$' 5-*GCL)7;;tR015=GKKQ'(1,   7?(+GDM(:J} % t zU2$' 5*7;;tR01&&7$%-GKKQ'(1,    t $(6*GCL)%7;;tR015=    t $GCL)   u :=.L6a8;w,ws|4A$!2+GCL9.GCL)7;;tR015=    u WS\"$' 5'#,''++c1-.5=    u WS\"$' 5'#,''++c1-.5=    =/*<mathceilappendzipextendranger) stack_argsrepeatsdepth_multiplier depth_truncrnum_repeat_scaledrepeats_scaledrVrs sa_scaledbareprDs r$_scale_stage_depthrsWJg 56F)F#GH  *7G*G HI N TrT]  E1z>,==? @b!a R  $DbD)NIz>2=Cc ;1(2,;<=  block kwargs Args: arch_def: architecture definition strings, list of list of strings depth_multiplier: network depth multiplier depth_trunc: networ depth truncation mode when applying multiplier experts_multiplier: CondConv experts multiplier fix_first_last: fix first and last block depths when multiplier is applied group_size: group size override for all blocks that weren't explicitly set in arch string Returns: list of list of block kwargs rhrrr}?) r~tupler enumeraterlistrrr setdefaultrr)arch_defrrexperts_multiplierfix_first_lastr} arch_args stack_idx block_stringsr2rrrrrs r$rrs_*I"E*#$H 555,.X>2;CJZ<[2\_. .M:-... & Ii- --' 2GBvvmQ'!+0BQ0F=!%77!% lJ7   b ! NN3   yA~c(ma>O1O   / GS+V W   / GZQ\] ^!_" r#ceZdZdZddeddddddddddf deded ed ed e e d e e d e e de e de de e defdZ dZ dZy)ra/ Build Trunk Blocks This ended up being somewhat of a cross between https://github.com/tensorflow/tpu/blob/master/models/official/mnasnet/mnasnet_models.py and https://github.com/facebookresearch/maskrcnn-benchmark/blob/master/maskrcnn_benchmark/modeling/backbone/fbnet_builder.py FNr_ output_stridepad_type round_chs_fn se_from_expr- norm_layeraa_layerse_layerdrop_path_ratelayer_scale_init_valuefeature_locationc||_||_||_||_||_||_||_t||_ |jddd|_ | |_ | |_ | dk(rtjdd} | |_| d vsJ| | d |_t"|_d|_g|_y#t$r d|_ YmwxYw) Nrrd_ratioTF depthwisez>feature_location=='depthwise' is deprecated, using 'expansion' expansion) bottleneckrr)devicedtype)rrrrr-rrr r se_has_ratio TypeErrorrrr6warningrdd_DEBUG_BUILDERverbosein_chsfeatures)selfrrrrr-rrrrrrrrs r$__init__zEfficientNetBuilder.__init__Es +  (&"$   *  & MM!cM * $D -&<# { * OO\ ]*  0#BBBB#e4%    & %D  &sB88C  C c\|j|z|z }|jd}|j|d<|j|d|d<|j dd}|dkDr |dxxdzcc<d|vr|dr|j|d|d<|j |d<|d |d n |j |d <|d J|j|d <||d <|j|j|d <|jd d}|rm|ja|js||j ddz}|dk(r|dz}|jrt|j||d<n|j|d<|j|j|dk(rYtdj!|t#||j$|j ddr t'd'i|n t)d'i|}n|dk(s|dk(r*??J ++I6 Ej$/ //F)2:)>@ %% :&]c*o= \00;T\\J!(+v555>+,Jx("z~~eU'C%h/1444"#K?"a'Jx,@A,E/0 8,";/14/&' ,7 5)#( %.]N'5=M9N'N(J(8;HEI%!1 h'!+)7*X:N)N&)D,>,>>(8:h;O(O /0 8,$%d%k%k ..&015?*<)9 :& $44'4$((_FWX e$!#"#K$#'$'!m"0$ ,,T-B-BC$L !- 0 02 >I 14WYKqQZP[;\^g:h1i X.))z3:9+1F X.MM((61$A@ %B MM"--0 1QH 2R k!CsL)__name__ __module__ __qualname____doc__rr>rrboolrrrrrrr"r#r$rr<s"$%3 %-1.2,0,0$&6:$&***# *  *  * *!+*y)*y)*"*%-UO*"*X>@`r#rct|tr|jd|jdz|jz|r|jzt fd|j |j}||j|j*tjj|jyyt|tjr|jd|jdz|jz|r|jztjj|jdtj dz |j*tjj|jyyt|tj"rStjj%|jtjj|jyt|tj&r|jj)dd}d|vr|jj)d}dtj |zz }tjj+|j| |tjj|jyy)a) Weight initialization as per Tensorflow official implementations. Args: m (nn.Module): module to init n (str): module name fix_group_fanout (bool): enable correct (matching Tensorflow TPU impl) fanout calculation w/ group convs Handles layers in EfficientNet, EfficientNet-CondConv, MixNet, MnasNet, MobileNetV3, etc: * https://github.com/tensorflow/tpu/blob/master/models/official/mnasnet/mnasnet_model.py * https://github.com/tensorflow/tpu/blob/master/models/official/efficientnet/efficientnet_model.py rrcrtjj|dtjdz S)Nr@)rinitnormal_rsqrt)wfan_outs r$z#_init_weight_goog..%s%bggooaDIIcGm,DEr#Nr routing_fnr)r~r ro out_channelsgroupsr rh weight_shapeweightbiasrr zeros_Conv2dr rr  BatchNorm2dones_Linearsizeuniform_)mrGfix_group_fanoutinit_weight_fnfan_in init_ranger s @r$_init_weight_googr!s!Z --"Q]]1%55F   G1 Eq}}VWVdVdfqxx 66  GGNN166 "  Aryy !--"Q]]1%55F   G !TYYsW}%=> 66  GGNN166 "  Ar~~ &  ahh qvv Aryy !((--" 1 XX]]1%F499Vg%566  J; ; qvv "r#modelc|xst}|jD]\}}|||t|D]"\}}t|ds|j$y)N init_weights)r!rhasattrr$)r"init_fnrGrs r$rr=s`**G##%11 e$1 1n % NN r#)rI)rrNg?)rr)rrrFN)rTr)2rtypingrrloggingrrHr!r functoolsrrrr torch.nnr timm.layersr r r r rr_efficientnet_blocks _manipulater__all__ getLoggerrr6rrrrr rrr%rrrr:rBrrrrr!Modulerr"r#r$r1s& ""pp#& t '  H %"28IJ d38n% & ;` /T"n#P +\UUp&RRYYr#