^j1 ddlZddlZddlZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZddlZddlZddlmZddlmZddlmZgdZd.d ej0d efd Z d/d e d ej0dededej0f dZ d/d ej0dededefdZ d/d ej0dededefdZdZ d0de eeefdee e fdedefdZ d0d ej0dedefdZ! d0d ej0dedefdZ" d1de eeej0fde#deeeed ffd!eeeeej0ffd"Z$dd#d$e efd%Z% d2d&e#d'ed(ed$e efd)Z&d*e#d+edefd,Z'd ej0de efd-Z(y)3N) defaultdict)chain) AnyCallableDictIteratorListOptionalTupleTypeUnion)nn)Tensor)use_reentrant_ckpt) model_parameters named_apply named_modulesnamed_modules_with_paramsadapt_input_convgroup_with_matcher group_modulesgroup_parametersflatten_modulescheckpoint_seq checkpointreinit_non_persistent_buffersmodel exclude_headcr|r!|jDcgc]}|c}ddS|jScc}w)N) parameters)rrps b/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/models/_manipulate.pyrrs: ++-.a.s33!!/s 4fnmodule depth_first include_rootreturnc|s |r ||||jD],\}}|rdj||fn|}t||||d.|r |r ||||S)N)r%name.T)r$r%r*r&r')named_childrenjoinr)r$r%r*r&r' child_name child_modules r#rrsp < &t$$*$9$9$;m L59SXXtZ01z r,Z[gklm| &t$ Mr*c#K|s|r||f|jD]3\}}|rdj||fn|}t|||dEd{5|r |r||fyyy7wNr+T)r%r*r&r')r,r-rr%r*r&r'r.r/s r#rr-s <Fl$*$9$9$;^ L59SXXtZ01z  jkX\^ ^ ^^|Fl${ ^sA A" A  A"c#K|jr |s|r||f|jD]3\}}|rdj||fn|}t|||dEd{5|jr |r |r||fyyyy7 wr2) _parametersr,r-rr3s r#rr=s +,Fl$*$9$9$;^ L59SXXtZ01z ,jkX\^ ^ ^^klFl/;k ^sAA;A9!A;)i named_objects group_matcher return_valuesreversecttrg}tjD]i\}\}}| t|tt fr+|D]%}|t j|d|f|dfgz }'M|t j||fdfgz }k|fd} tt } |D]#\} } | | | j|r| n| %tt } d}ttd| jD]2} |dks| dtdk7r|dz }| |j| | 4|r1|rJdi}| jD]\}}|D]}|||< |S| S)Nrc tttfruD]d\}}}|j|}|s||j |f}tt t tjtd|cSt dfS|}t|tjjs|fSt|S)Ninf) isinstancelisttuplematchgroupsmapfloatr from_iterablefilter collectionsabcIterable)r*match_fnprefixsuffixrpartsordr7s r# _get_groupingz)group_with_matcher.._get_groupinges mdE] 3,9 W(&&NN4(#QXXZ8E UE,?,?tU@S,T!UVV  W <= %Cc;??#;#;<t : r0c |duS)N)xs r#z$group_with_matcher..|s Qd]r0z-reverse mapping only sensible for name output)r>dict enumerateitemsr@r?recompilerappendsortedrFkeysMATCH_PREV_GROUPextend)r6r7r8r9compiled group_ordinal group_namemspecsspecrPgroupingkvlayer_id_to_paramlidparam_to_layer_idlmns ` r#rrPs -&2;Mr@strr Sequential ModuleList ModuleDictrr,r-)rrrrKrsprefix_is_tupler*r%s r#rrs !/O,$ ; &MM2=="--HLMM+L%# f Z 5&%%' "1wt)   'Fl"88VTN3DFl"# sB%C'C(3C use_reentrantr|cx| t}tjjj|g|d|i|S)z~ checkpoint wrapper fn A thin wrapper around torch.utils.checkpoint.checkpoint to default use_reentrant to False r|)rtorchutilsr)functionr|argskwargss r#rrsN*, ;; ! ! , ,   $    r0everyflatten skip_lastc| t}d}t|tjjr|j }|rt j|}t|ttfs t|}t|}|r|dz}d}td||D]K} t| |zdz |dz }tjjj|| ||||}M|r||dzt|dz ||S|S)a,A helper function for checkpointing sequential models. Sequential models execute a list of modules/functions in order (sequentially). Therefore, we can divide such a sequence into segments and checkpoint each segment. All segments except run in :func:`torch.no_grad` manner, i.e., not storing the intermediate activations. The inputs of each checkpointed segment will be saved for re-running the segment in the backward pass. See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works. .. warning:: Checkpointing currently only supports :func:`torch.autograd.backward` and only if its `inputs` argument is not passed. :func:`torch.autograd.grad` is not supported. .. warning: At least one of the inputs needs to have :code:`requires_grad=True` if grads are needed for model inputs, otherwise the checkpointed part of the model won't have gradients. Args: functions: A :class:`torch.nn.Sequential` or the list of modules or functions to run sequentially. x: A Tensor that is input to :attr:`functions` every: checkpoint every-n functions (default: 1) flatten: flatten nn.Sequential of nn.Sequentials skip_last: skip checkpointing the last function in the sequence if True use_reentrant: Use re-entrant checkpointing Returns: Output of running :attr:`functions` sequentially on :attr:`*inputs` Example: >>> model = nn.Sequential(...) >>> input_var = checkpoint_seq(model, input_var, every=2) cfd}|S)NcFtdzD] }||}|S)Nr;)range)_xjend functionsstarts r#forwardz5checkpoint_seq..run_function..forwards25#'* &!Yq\"% &Ir0rS)rrrrs``` r# run_functionz$checkpoint_seq..run_functions r0r;rQrr{)rr>r~rrwchildrenrrEr@r?lenrminrr) rrTrrrr|rnum_checkpointedrrs r#rrsV*, )UXX001&&( '' 2 i% /)$ 9~A Cq*E2 %%-!#%5%9: KK " " - - Y / ' .  C|C!GS^a%7CAFF Hr0in_chans conv_weightch|j}|j}|jdk7rtd|jd|j\}}}}|dk(r\|dkDrC|jddzdk(sJ|j ||dzd||}|j dd }n~|j dd }nj|dk7re|dk7r td ttj|dz }|jd|ddddd|ddddf}|dt|z z}|j|}|S) Nz;adapt_input_conv only supports 4D Conv2d weights; got ndim=r+r;rF)dimkeepdimTz*Weight format not supported by conversion.) dtyperDndimNotImplementedErrorshapereshapesumintmathceilrepeatto)rr conv_typeOIJKrs r#rr!sO!!I##%K1"I+JZJZI[[\ ]  ""JAq!Q1} q5$$Q'!+q0 00%--aaAqAK%//a/?K%//a/>K Q 6%&RS S8a<01F%,,Q1=a(Aq>PQK Ah/ 0K..+K r0cg}|jD]7\}}t|ds|j|j|r|nd9|S)aWalk model and call init_non_persistent_buffers() on modules that have it. This reinitializes computed buffers (like RoPE frequencies, attention bias indices) that are marked as non-persistent and thus not saved in checkpoints. These buffers are typically computed from module configuration and need to be reinitialized after loading a checkpoint. Args: model: Model to reinitialize buffers for Returns: List of module names that were reinitialized Example: >>> model = create_model('vit_base', pretrained=True) >>> # After loading checkpoint or moving to new device >>> reinitialized = reinit_non_persistent_buffers(model) >>> print(f"Reinitialized {len(reinitialized)} modules") init_non_persistent_buffersz(root))rhasattrrr[)r reinitializedr*r%s r#rrAsV(M++-= f 68 9  . . 0  8 <= r0)F)TF)FF)r;r sequential)r;FFN))collections.abcrGrrYr itertoolsrtypingrrrrr r r r r r~torch.utils.checkpointrr timm.layersr__all__Moduleboolrrrvrrr^rrrrrrrrrrSr0r#rs #TTT * , "BII"T"!# "        YY $ "      $ "        $ :c3h0:T8^,:: :@$ ` `` `$ h hh h.0;G #c299n 56##c5c?*+#CtBII!778 #B%)D>2(, I I  I  I  ~ I XsF@tCyr0