^j9R fdZddlmZddlmZmZmZmZmZm Z ddl Z ddl m Z ddl mZmZddlmZmZmZmZmZmZmZmZddlmZdd lmZdd lmZdd lm Z m!Z!d gZ"Gd de jFZ$Gdde jFZ%de&fdZ'Gdde jFZ(Gdde jRZ*Gdd e jFZ+d+dZ,dZ-d,dZ.d-dZ/e!e/de/de/d e/d!e/d"d#dd$%d&Z0e d,d'Z1e d,d(Z2e d,d)Z3e d,d*Z4y).z? RDNet Copyright (c) 2024-present NAVER Cloud Corp. Apache-2.0 )partial)ListOptionalTupleUnionCallableTypeNIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)DropPathcalculate_drop_path_ratesNormMlpClassifierHeadClassifierHeadEffectiveSEModulemake_divisible get_act_layerget_norm_layer)build_model_with_cfg)feature_take_indices) named_apply)register_modelgenerate_default_cfgsRDNetc teZdZ ddedededeej deej f fd ZdZxZ S) Blockin_chs inter_chsout_chs norm_layer act_layerc ||d}t |tjtj||f|dddd|||fi|tj||fdddd||tj||fdddd||_yNdevicedtyper)groups kernel_sizestridepaddingrr+r,r-)super__init__nn SequentialConv2dlayers selfrrr r!r"r&r'dd __class__s \/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/models/rdnet.pyr0zBlock.__init__s / mm IIff ^V1VW ^[] ^ v $ $ IIfi RQq! Rr R K IIi Sa1 SPR S   c$|j|SNr4r6xs r9forwardz Block.forward-{{1~r:NN __name__ __module__ __qualname__intr r1Moduler0r@ __classcell__r8s@r9rrsU     RYY  BII  (r:rc teZdZ ddedededeej deej f fd ZdZxZ S) BlockESErrr r!r"c .||d}t |tjtj||f|dddd|||fi|tj||fdddd||tj||fdddd|t |fi||_yr$)r/r0r1r2r3rr4r5s r9r0zBlockESE.__init__2s / mm IIff ^V1VW ^[] ^ v $ $ IIfi RQq! Rr R K IIi Sa1 SPR S g , ,   r:c$|j|Sr<r=r>s r9r@zBlockESE.forwardGrAr:rBrCrJs@r9rLrL1sU     RYY  BII  *r:rLblockc|jj}|dk(rtS|dk(rtSJd|d)NrOblockesezUnknown block type (z).)lowerstriprrL)rOs r9_get_block_typerTKsD KKM   !E  * 6,UG266ur:ceZdZdddddddddejej ddf ded ed ed ed ed edededede ejde ejffd Z de e jde jfdZxZS) DenseBlock@@rrư>Nnum_input_features growth_ratebottleneck_width_ratiodrop_path_rate drop_raterand_gather_step_prob block_idx block_type ls_init_valuer!r"c x| | d}t|||_||_||_||_||_| dkDr,tj| tj|fi|znd|_ t|}t||zdz dz}t||_t|d|||| | d||_y)Nr%r)rrr r!r")r/r0r_r^r`rar\r1 ParametertorchonesgammarGr drop_pathrTr4)r6r[r\r]r^r_r`rarbrcr!r"r&r'r7rr8s r9r0zDenseBlock.__init__Vs  / ",%:""&TadeTeR\\-%**[2OB2O"OPko +& *-CCaGH1L !.11oj1 %!     r:r?returnctj|d}|j|}|j-|j |jj dddd}|j |}|S)Nr)rhcatr4rjmulreshaperkr>s r9r@zDenseBlock.forward}s^ IIaO KKN :: !djj((B156A NN1 r:)rDrErFr1 LayerNormGELUrGfloatstrr rHr0rrhTensorr@rIrJs@r9rVrVUs')!,/$'"+.%#'*,,,)+% #% % %* % " %  % $)% % % !% RYY% BII% Nell+ r:rVc teZdZ d dededeedeffd ZdejdejfdZ xZ S) DenseStage num_blockr[drop_path_ratesr\c ||d}t |t|D]1} td|||| | d||} ||z }|j d| | 3||_y)Nr%)r[r\r^ra dense_blockrf)r/r0rangerV add_modulenum_out_features) r6ryr[rzr\r&r'kwargsr7ilayerr8s r9r0zDenseStage.__init__s / y! 6A#5'.q1     E + -  OOk!-u 5 6!3r: init_featurerlct|g}|D]}||}|j|tj|dS)Nr)appendrhro)r6rfeaturesmodule new_features r9r@zDenseStage.forwardsA > )F *K OOK ( )yy1%%r:rB) rDrErFrGrrtr0rhrvr@rIrJs@r9rxrxsU33!$3"%[ 3  32&ELL&U\\&r:rxc+eZdZ d*dedededeeeeefdeeeeefdeeeeefdeeeeefde d e d e d ed ed ede dededeee fdede e de de f*fd Z e jjd+dZe jjd,dZe jjdej&fdZd-dede efdZ d.de j,de eeeefdeded ed!edeee j,ee j,ee j,fffd"Z d/deeeefd#ed$efd%Zd&Zd+d'efd(Zd)ZxZS)0rin_chans num_classes global_pool growth_ratesnum_blocks_listrbis_downsample_blockr]transition_compression_ratiorc stem_type patch_sizenum_init_featureshead_init_scalehead_norm_first conv_biasr"r!norm_epsr_r^ct%|||d}t|t|cxk(rt|k(sJJt|}t |}| t ||}||_||_||_| dvsJ| dk(r>tjtj|| f| | |d||| fi||_ | }nmd| vrt| dzn| }tjtj||fd dd |d |tj|| fd dd |d ||| fi||_ d }g|_t||_|}| }t!||d }g}t#|jD]+}g} |dk7rjt%|| zdz dz}!d x}"}#||r |dz}dx}"}#| j'||fi|| j'tj||!f|"|#dd||!}t)d|||||||||| ||||d |}$| j'|$|||||zz }|d z|jk(s|d z|jk7r1||d zr)|xjt+||d|||gz c_|j'tj| .tj||_|x|_|_|rB||j.fi||_t5|j.|f||jd||_nCtj8|_t;|j.|f||j|d||_t=t t>||y)a  Args: in_chans: Number of input image channels. num_classes: Number of classes for classification head. global_pool: Global pooling type. growth_rates: Growth rate at each stage. num_blocks_list: Number of blocks at each stage. is_downsample_block: Whether to downsample at each stage. bottleneck_width_ratio: Bottleneck width ratio (similar to mlp expansion ratio). transition_compression_ratio: Channel compression ratio of transition layers. ls_init_value: Init value for Layer Scale, disabled if None. stem_type: Type of stem. patch_size: Stem patch size for patch stem. num_init_features: Number of features of stem. head_init_scale: Init scaling value for classifier weights and biases. head_norm_first: Apply normalization before global pool + head. conv_bias: Use bias layers w/ all convolutions. act_layer: Activation layer type. norm_layer: Normalization layer type. norm_eps: Small value to avoid division by zero in normalization. drop_rate: Head pre-classifier dropout rate. drop_path_rate: Stochastic depth drop rate. r%N)eps)patchoverlapoverlap_tieredr)r+r,biastieredr)r)r+r,r-rT) stagewiserrer.) ryr[r\r]r_rzrcrbr!r"z dense_stages.)num_chs reductionrr\) pool_typer_)rr_r!)rrf) r/r0lenrrrrrr_r1r2r3stemr feature_info num_stagesrr}rGrrxdict dense_stages num_featureshead_hidden_sizenorm_prerheadIdentityrr _init_weights)&r6rrrrrrbrr]rrcrrrrrrr"r!rr_r^r&r'r7 stem_stridemid_chs curr_striderdp_ratesrrdense_stage_layerscompressed_num_featuresk_sizer,stager8s& r9r0zRDNet.__init__sb  /< C$8TC@S\}}||}|| vs|r|| k(r|j|}n|}|j|@|r|S| k(r|j|}||fScc}wcc} w)a Forward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features )NCHWzOutput shape must be NCHW.r.rnrN) rrGsplitrrrrrhjit is_scripting enumeraterr)r6r?rrrrr intermediatesinfo stage_ends take_indices max_indexrlast_idxrfeat_idxrx_inters r9forward_intermediateszRDNet.forward_intermediates`se&Y&D(DD& EIEVEVWTc$x...s3B78W W"6s:"P i/;s r9forward_featureszRDNet.forward_featuress2 IIaL   a  MM! r: pre_logitscN|r|j|dS|j|S)NT)r)r)r6r?rs r9 forward_headzRDNet.forward_heads$0:tyyty,L ! Lr:cJ|j|}|j|}|Sr<)rrr>s r9r@z RDNet.forwards'  ! !! $   a r:)r)avg)rWhrrr)r)r)r)r)r)r)r))rrrLrLrLrLrLNTTFFFTrX?rZrrrW?FTgelu layernorm2dNrYrYNNF)Tr<)NFFrF)rFT)rDrErFrGrurrrboolrtrrr0rhrignorerrr1rHrrrvrrrrr@rIrJs@r9rrs #$9[!?^S%*^S+0^S!^S^S^S #^S#^S "!^S"#^S$S(]+%^S&'^S(uo)^S*+^S,"-^S@ YY   YY** YY 2C2hsm28<$$',1 ||1 eCcN341  1  1  1 !%1  tELL!5tELL7I)I#JJ K1 j./$# 3S >* $ M$Mr:rct|tjr*tjj |j yt|tj rUtjj|j dtjj|jdyt|tjr}tjj|jd|rPd|vrK|j jj||jjj|yyyy)Nrrzhead.) isinstancer1r3initkaiming_normal_weight BatchNorm2d constant_rLineardatamul_)rnamers r9rrs&"))$  . FBNN + &--+ &++q) FBII & &++q) GtO MM   # #O 4 KK   ! !/ 2$4 'r:cd|vr|Sd|vr|d}i}|jD]\}}|jdd}|||<|S)z Remap NV checkpoints -> timm z stem.0.weightmodelz stem.stem.zstem.)itemsreplace) state_dictrout_dictkvs r9checkpoint_filter_fnrsc*$*( H  "1 IIlG ,  Or:c Ntt||fttddd|}|S)N)rrrr)T) out_indicesflatten_sequential)pretrained_filter_fn feature_cfg)rrrr)variant pretrainedrrs r9 _create_rdnetrs6  w  1\dK    E Lr:c 8|dddddttdddd d d d |S) Nr)r)rr)r(r(g?bicubiczstem.0zhead.fczarXiv:2403.19588z:DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTsz!https://github.com/naver-ai/rdnetz apache-2.0)urlr input_size pool_sizecrop_pct interpolationmeanstd first_conv classifier paper_ids paper_name origin_urllicenser )rrs r9_cfgr s<=v)%.Bi'R9    r:znaver-ai/rdnet_tiny.nv_in1k) hf_hub_idznaver-ai/rdnet_small.nv_in1kznaver-ai/rdnet_base.nv_in1kznaver-ai/rdnet_large.nv_in1kz(naver-ai/rdnet_large.nv_in1k_ft_in1k_384)r)r") r#)r!rrr)zrdnet_tiny.nv_in1kzrdnet_small.nv_in1kzrdnet_base.nv_in1kzrdnet_large.nv_in1kzrdnet_large.nv_in1k_ft_in1k_384c d}ddgdgzdgdzzdgzdg|zdd d gd gzd gdzzd gzd }tdd |it|fi|}|S)Nr(rWrrrrr)rrrrLrrrrrrbr) rdnet_tinyrrrrn_layer model_argsrs r9r&r&sGu uqy0C583=L(+i7)+zlQ.>>*M J  \: \jA[TZA[ \E Lr:c d}ddgdgzdg|dz zzdgdzzdg|zd d d gd gzd g|dz zzd gdzzd }tdd|it|fi|}|S)N HrWrrrr) NTTFFFFFFTFrrrLr%r) rdnet_smallr'r(s r9r0r0sGu u! '<<uqyH3=h(+i7)+zlgk.JJj\\]M]] J  ]J ]$zB\U[B\ ]E Lr:c d}ddgdgzdg|dz zzdgdzzd g|zd d d gd gzd g|dz zzd gdzzd}tdd|it|fi|}|S)Nr,x`rriPrr)r/rrrLr%r) rdnet_baser'r(s r9r5r5sG u u! '<<uqyH3=h(+i7)+zlgk.JJj\\]M]] J  \: \jA[TZA[ \E Lr:c d}ddgdgzdg|dz zzdgdzzd g|zd d d gd gzd g|dz zzd gdzzd}tdd|it|fi|}|S)Nr#rrihrr)) NTTFFFFFFFTFrrrLr%r) rdnet_larger'r(s r9r:r:%sG  1(== I3=o(+i7)+zlgk.JJj\\]M]] J  ]J ]$zB\U[B\ ]E Lr:)Nrr)r)5__doc__ functoolsrtypingrrrrrr rhtorch.nnr1 timm.datar r timm.layersr rrrrrrr_builderr _featuresr _manipulater _registryrr__all__rHrrLrurTrVr2rxrrrrr  default_cfgsr&r0r5r:rfr:r9rGsU ?? A222*+$< )BII2ryy473700f&&DGBIIGT 3   %/102/102'+< 3((D &          r: