^j'#dZddlmZddlmZmZmZddlZddlm Z ddl m Z m Z ddl mZGdde jZy) zBase training task abstraction. This module provides the base TrainingTask class that encapsulates a complete forward pass including loss computation. Tasks return a dictionary with loss components and outputs for logging. ) nullcontext)AnyDictOptionalN)get_state_dict unwrap_model) ModelEmaV3c @eZdZdZ d(deej deejdeffd Z fdZ d)dee d dfd Z d*d e d ee d eejfd Zd+ded eejfdZ d,dededeej d ejfdZd efdZ d*d e d ee d eejfdZd)deed dfdZd+defdZd-deejded eejfdZd-deejded ee effdZ d.deee efdedeejded df dZdefded ee effd Z d/d!ee efd"eee efdeded df d#Z d$Z!d%ejDd&ejDd ee ejDffd'Z#xZ$S)0 TrainingTaskaBase class for training tasks. A training task encapsulates a complete forward pass including loss computation. Tasks return a dictionary containing the training loss and other components for logging. The returned dictionary must contain: - 'loss': The training loss for backward pass (required) - 'output': Model output/logits for metric computation (recommended) - Other task-specific loss components for logging (optional) Args: device: Device for task tensors/buffers (defaults to cpu) dtype: Dtype for task tensors/buffers (defaults to torch default) verbose: Enable info logging Example: >>> task = SomeTask(model, criterion, device=torch.device('cuda')) >>> >>> # Prepare for distributed training (if needed) >>> if distributed: >>> task.prepare_distributed(device_ids=[local_rank]) >>> >>> # Training loop >>> result = task(input, target) >>> result['loss'].backward() Ndevicedtypeverbosect|||ntjd|_||ntj|_||_y)Ncpu)super__init__torchr get_default_dtyper r)selfr r r __class__s Y/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/task/task.pyrzTrainingTask.__init__-sE  & 2f U8K #/UU5L5L5N  ctjdj|i|}|j|_|j|_t ||i|S)zFMove task to device/dtype, keeping self.device and self.dtype in sync.r)remptytor r r)rargskwargsdummyrs rrzTrainingTask.to8sL! A!!4262ll [[ wz4*6**r device_idsreturnc |S)aPrepare task for distributed training. This method wraps trainable components in DistributedDataParallel (DDP) while leaving non-trainable components (like frozen teacher models) unwrapped. Should be called after task initialization but before training loop. Args: device_ids: List of device IDs for DDP (e.g., [local_rank]) **ddp_kwargs: Additional arguments passed to DistributedDataParallel Returns: self (for method chaining) Example: >>> task = LogitDistillationTask(student, teacher, criterion) >>> task.compile() >>> task.prepare_distributed(device_ids=[args.local_rank]) )rr ddp_kwargss rprepare_distributedz TrainingTask.prepare_distributed?s 2 rbackendmodec y)a'Compile hot task components before distributed wrapping. Subclasses should compile the train/eval modules that do the tensor work, not the outer task wrapper. The return value is the eval-facing compiled module/callable used by validation and checkpoint export. Nr"rr%r&compile_kwargss rcompilezTrainingTask.compileZsrFemac:|r t|ddSt|d|S)z1Return the module that owns trainable parameters.trainable_module_emaNtrainable_module)getattr)rr+s rget_trainable_modulez!TrainingTask.get_trainable_modulehs% 4!7> >t/66rdecay use_warmupc `t|jf|||d||_|jS)z+Create an EMA copy of the trainable module.)r1r2r )r r0r-)rr1r2r rs r setup_emazTrainingTask.setup_emansC%/  % % '% ! %  % !(((rc*|jdduS)z5Return whether this task has an EMA trainable module.Tr+N)r0)rs rhas_emazTrainingTask.has_emas((T(2$>>rc |jsytj|jdf||d||_|jS)z3Compile the EMA eval model if one has been created.NTr6)r%r&)r7rr*get_eval_modeleval_model_emar(s r compile_emazTrainingTask.compile_emasU||~#mm   D  )     """rstepc|jr;|jdjt|j|yy)z3Update EMA state from the current trainable module.Tr6)r<N)r7r0updater)rr<s r update_emazTrainingTask.update_emas@ <<>  % %$ % / 6 6|DD]D]D_7`gk 6 l r exclude_headc|j}|r!|jDcgc]}|c}ddS|jScc}w)z/Return parameters to use for gradient clipping.N)r0 parameters)rr@r.ps rget_clip_parametersz TrainingTask.get_clip_parameterssI446 /::<=!A=crB B**,,>s Amodulecj|0|rdnd}t||r t||S|j|}|S)zReturn the eval model/callable used for validation. Checkpoint state_dict handling uses unwrap_model separately so DDP and compiled wrappers do not leak into saved keys. r: eval_modelr6)hasattrr/r0)rrFr+ eval_attrs rr9zTrainingTask.get_eval_modelsB >,/(\ItY'tY//..3.7F rciS)z:Return task-owned state outside the eval model state_dict.r")rrFr+s rget_task_statezTrainingTask.get_task_states rstatestrictcy)z8Load task-owned state outside the eval model state_dict.Nr")rrMrNrFr+s rload_task_statezTrainingTask.load_task_statesrc|j|}|iS|rdnd}|rdnd}|j|}|t||i}|r|||<|S)z3Return checkpoint state entries owned by this task.r6state_dict_ema state_dicttask_state_ema task_state)r9rLr)rr+ unwrap_fnrH model_keytask_keyrUrMs rget_checkpoint_statez!TrainingTask.get_checkpoint_statesk ((S(1  I(+$ '*# ((S(1 N:yAB (E(O rrSrUc|j|}||r tdtdt|j|||j |||y)z-Load model and task-owned checkpoint entries.r6Nz4Cannot load EMA checkpoint state before setup_ema().z3Cannot load checkpoint state without an eval model.)rN)rNr+)r9 RuntimeErrorrload_state_dictrP)rrSrUr+rNrHs rload_checkpoint_statez"TrainingTask.load_checkpoint_statese((S(1  "#YZZTU UZ 00F0K ZC@rcz|j}| ||urt|dr|jStS)zReturn a no-sync context for gradient accumulation. Tasks that wrap a trainable component with DDP delegate to that component's no_sync(). Non-distributed tasks use a no-op context. no_sync)r0rIr_r)rrFs rr_zTrainingTask.no_syncs= **,  &"49S>># #}rinputtargetct)zPerform forward pass and compute loss. Args: input: Input tensor [B, C, H, W] target: Target labels [B] Returns: Dictionary with at least 'loss' key containing the training loss )NotImplementedError)rr`ras rforwardzTrainingTask.forwards "!r)NNT)N)inductorN)F)gH.?FN)NF)TNF)NFT)%__name__ __module__ __qualname____doc__rrr r boolrrlistr$strnnModuler*r0floatr4r7r;intr?rEr9rrrLrPrrYr]r_Tensorrd __classcell__)rs@rr r s:.2+/ U\\* EKK(  +*.   :&"&  3-  "))  77"))9L7"$-1 )))U\\* )  )"?? &"&##3-# "))  #"mx}mm -- Xbii%8 d W_`b`i`iWj Xbii%8dW[\_ad\dWe *. DcN+RYY'     " c3h ,48 AS#XA!c3h0A A  A  A "<<"LL" c5<< "rr )ri contextlibrtypingrrrrtorch.nnrmtimm.utils.modelrrtimm.utils.model_emar rnr r"rrrxs3 #&& 9+f"299f"r