^jƎ UdZddlmZddlZddlZddlZddlmZmZm Z m Z ddl Z ddl m cmZddlmZmZddlmZddlmZmZddlmZdd lmZmZdd lmZmZm Z dd l!m"Z"dd l#m$Z$e$Z%d ddddddddd Z&de'd<GddeZ(y)z4LightningModule for RF-DETR training and validation.) annotationsN)AnyDictOptionalTuple)LightningModuleseed_everything)_namespace_from_configs) ModelConfig TrainConfig)compute_multi_scale_scales)build_criterion_from_configbuild_model_from_config) apply_lorainterpolate_position_embeddingsload_pretrain_weights)get_param_dict) get_loggerloss_clsloss_box loss_gioumask_ce mask_dicekp_l1kp_findkp_viskp_nll) loss_ce loss_bboxr loss_mask_celoss_mask_diceloss_keypoints_l1loss_keypoints_findableloss_keypoints_visibleloss_keypoints_nllzdict[str, str]_TRAIN_PROGRESS_LOSS_ALIASESc2eZdZdZdfd ZddZddZddZddZ ddZ ddZ dd Z dd Z d d Z dd Zed!d Z d"dZ d"dZd#dZed$dZd%dZ d& d'fd Zd#dZd(d)dZd*dZd+dZxZS),RFDETRModelModulezLightningModule wrapping the RF-DETR model and training loop. Args: model_config: Architecture configuration. train_config: Training hyperparameter configuration. c,t|||_||_t t |dd|_|j |_d|_d|_ t|||_ |jjt|j|j|jr>t |jdd}t|r|t j#d|j$rt'|jt)|j|j\|_|_ddlm}t3|j4j7}|dv}|j8xr|dk(xr|xr |j: }|j8r!|j:rt j#d |red t<j>j@_!d t<j>j@_"t=j8|jd |_ yy) Nuse_grouppose_keypointsF"reset_keypoint_gaussian_parameterszVReset keypoint Gaussian precision outputs to unit values after pretrained weight load.r)DEVICE>gpuautocudar/zQDisabling torch.compile because multi_scale=True introduces dynamic input shapes.T)dynamic)#super__init__ model_config train_configboolgetattr_use_manual_optimizationautomatic_optimization_accumulated_box_normalizerstrict_loadingrmodelpretrain_weightsrr*callableloggerinfo backbone_lorarr criterion postprocess rfdetr.configr,str acceleratorlowercompile multi_scaletorch_dynamoconfigsuppress_errorscapture_scalar_outputs) selfr3r4r+r,rEuses_cuda_acceleratorcompile_enabled __class__s g/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/rfdetr/training/module_model.pyr2zRFDETRModelModule.__init__2s (( /37?68KKp  % % tzz ",GtGXGXZ^ZkZk+l(( ),22399; +/F F  pVv%5 p:O pXdXpXpTp    L$<$< KKk l 48EMM 0:>EMM 7tzz4@DJ c|jj/t|jj|jzdyy)zSeed RNGs at fit start when ``TrainConfig.seed`` is set. This avoids hidden global side-effects in ``build_trainer`` while still preserving deterministic training behaviour for actual fit runs. NT)workers)r4seedr global_rankrNs rR on_fit_startzRFDETRModelModule.on_fit_startws<    ! ! - D--22T5E5EEt T .rSc|j}|j}|jr0|js"|\}}t |j |j |j|j}|jj}tj|tj|} tj5t!j"|j$| dd|_t!j"|j&j)dj+| dj-dj/|_dddyyy#1swYyxYw)anApply optional multi-scale resize to the incoming batch. Modifications to ``batch`` (in-place on ``NestedTensor``) are visible in ``training_step`` because they share the same object. Args: batch: Tuple of (NestedTensor samples, list of target dicts). batch_idx: Index of the current batch within the epoch. bilinearF)sizemode align_cornersnearest)r\r]N)r4r3rHdo_random_resize_via_paddingr resolutionexpanded_scales patch_size num_windowstrainer global_steprandomrVchoicerIno_gradF interpolatetensorsmask unsqueezefloatsqueezer5) rNbatch batch_idxtcmcsamples_scalesstepscales rRon_train_batch_startz&RFDETRModelModule.on_train_batch_starts      >>""A"AJGQ/ r?Q?QSUS`S`bdbpbpqF<<++D KK MM&)E "#--eR\lq"rMM',,"8"8";"A"A"C%V_`hhijkppr    #B>   s B EE c|jD |j}t|tr|n|gD]}|j  d|_yd|_y#t $r Yd|_ywxYw)ayReset the accumulated box normalizer at the start of every training epoch. Lightning may reuse the module across epochs without calling ``_step_optimizer`` at the boundary (for example when an epoch ends mid-accumulation window with a non-divisible batch count). Clearing the accumulator here guarantees the manual-optimization path always starts each epoch from a known state, so the first microbatch's gradients are scaled by its own box count and not by a stale previous-epoch denominator. This is a no-op for non-keypoint models because they use Lightning's automatic optimization path and never populate ``self._accumulated_box_normalizer``. Note: on finite datasets the final-batch fallback in ``_should_step_optimizer`` always flushes a partial trailing window, so this reset is the only change needed. On IterableDatasets (infinite ``num_training_batches``) a partial window may survive epoch end with un-stepped gradients; those are discarded here and the optimizer is zeroed so the first microbatch of the new epoch starts from a clean state. N)r9 optimizers isinstancelist zero_grad RuntimeError)rNoptsopts rRon_train_epoch_startz&RFDETRModelModule.on_train_epoch_startss  + + 7 (#-dD#94v$CMMO$,0(4(  +/( s:A A,+A,c V|\}}t|}|j||}|jr)|j||\}}|j ||} n|j ||d} |j j tfdD} tdt|jj} |jr| n| | z } t|jj} t|jj}|j!j#Dcic] \}}d|| c}}|d| ||j%d| d|d| | |j'| | |j)}t+|t,r|d }|j.Dcgc] }d |vs|d }}|r]|d }t1|}t|}|j%d |ddd|j%d|ddd|j%d|ddd|jr3|j3| |j5|r|j7||jj8rt;j<5t;j>|Dcgc]}|d c}}|j@jB}|j#Dcic]@\}}|dvr7t+|t:jDr||jFdk\r |ddd|fn|B}}}|jI||}ddd|jr| jKn| |jM|dS|jr| jKS| Scc}}wcc}wcc}wcc}}w#1swYnxYw)a Compute loss for one training step and log metrics. PTL handles AMP (``precision``) without a manual ``GradScaler``. Keypoint models perform manual optimization so box-count loss normalization is based on the full accumulated effective batch rather than each microbatch independently; detection and segmentation models keep Lightning's automatic optimization path. Args: batch: Tuple of (NestedTensor samples, list of target dicts). batch_idx: Batch index within the epoch. Returns: Scalar loss tensor by default. When ``compute_train_metrics=True``, returns a Lightning-compatible dict containing ``loss`` plus detached postprocessed predictions for train mAP logging. Nc3>K|]}|vs||zywN.0k loss_dict weight_dicts rR z2RFDETRModelModule.training_step..s%XQqKGW9Q<+a.0X r_ztrain/Ton_stepon_epoch sync_dist batch_sizez train/lossF)prog_barrrrrrrlrztrain/lr)rrrz train/lr_minz train/lr_max orig_size) pred_logits pred_boxes pred_maskspred_keypoints)lossresultstargets)'lenr;r7_compute_train_losses_scale_loss_for_accumulationrArsummaxintrfaccumulate_grad_batchesr5r4train_log_sync_disttrain_log_on_steplog_dictitemslog_log_train_progress_metricsr}r~r param_groupsminmanual_backward_should_step_optimizer_step_optimizercompute_train_metricsrIrjstackr3 num_queriesTensorndimrBdetach_detach_results)rNrrrsrvrroutputsraw_loss normalizerloss_for_backwardrrloss_for_returnrrrv optimizerpg group_lrsbase_lrmin_lrmax_lrt orig_sizesnqinference_outputsrrrs @@rR training_stepzRFDETRModelModule.training_steps !\ **Wg.  ( (.2.H.HRY.Z +Ix $ A A(J W w8I $ nn00 X)XX#&aT\\-Q-Q)R"S"&"?"?$TLcEc"4#4#4#H#HI !2!2!D!DE )2): ;AvaS\1_ ;%)!     %)!   ((yZ(POO% i &!! I)2(>(>M"$"*RXM M lG^F^F HHZ5$QVH W HH^VeTTYH Z HH^VeTTYH Z  ( (  !2 3**95$$Y/    2 2 J"[[')JQ!K.)JK &&22 !( %1YY"1ell3AFFaKqCRCyQ6%!% **+K|]}|vs||zywrr)rrnumerator_loss_dictrs rRrz:RFDETRModelModule._compute_train_losses..4s)p1_`do_o*1- A>pr) rArr6 ValueErrortype__name__num_boxes_for_targetsrI ones_likerr) rNrrrkeyvaluerrrrs @@rRrz'RFDETRModelModule._compute_train_lossessnn00 t~~'JER'0012^^  ^^99'7K "nnWgYcIdneXkWpWpWr ISeCsk'9#u D  pH[pp(J..  s1C$c|j}|j}||n||z}||j||z |j|_||z S)a$Scale the current numerator loss by the accumulated box denominator. Args: raw_loss: Current microbatch weighted loss numerator. normalizer: Current microbatch box denominator. Returns: Loss scalar to pass to ``manual_backward``. )rr9_rescale_accumulated_gradients)rNrrprevious_normalizeraccumulated_normalizers rRrz.RFDETRModelModule._scale_loss_for_accumulation7sj &&( ">>/B/JPcfpPp  *  / /0CF\0\ ]+A+H+H+J(000rSc|jD]c}|j|jj|j|jj|jj ey)zRescale gradients already accumulated in the current optimizer window. Args: scale: Multiplicative factor that converts previous gradients from the old denominator to the new one. N)devicedtype) parametersgradmul_torr)rNrz parameters rRrz0RFDETRModelModule._rescale_accumulated_gradientsMs\ * hI~~)##EHHINN4I4IQZQ_Q_QeQeH$fg hrSctdt|jj}|dz|zdk(ryt |j dd}t |ttfxrtj|xr|dz|k\S)uReturn whether the current batch closes an optimizer accumulation window. The optimizer steps when either: - The current batch closes a complete ``grad_accum_steps`` window (``(batch_idx + 1) % grad_accum_steps == 0``), or - This is the final batch of the epoch and a partial accumulation window is still open, so the trailing microbatches are not silently dropped. Lightning's ``Trainer.num_training_batches`` may be reported as ``float('inf')`` for iterable / streaming datasets where the epoch length is unknown. In that case only the modulo path can ever close the window — the final-batch fallback is skipped because ``batch_idx + 1`` can never reach infinity. Args: batch_idx: Batch index within the epoch. Returns: ``True`` when the optimizer should step after this batch. r_rTnum_training_batchesN) rrr4grad_accum_stepsr6rfr~rpmathisfinite)rNrs accum_stepsrs rRrz(RFDETRModelModule._should_step_optimizerWs*!S!2!2!C!CDE M[ (A -&t||5KTR +c5\ : 6 23 6A !55 rSct|jdd}||jj}nt |t t fr|}nd}t|jdd}t |tsd}||dkDr|j||||j|j|jd|_ y)zClip gradients, step optimizer and scheduler, then reset accumulation state. Args: optimizer: Optimizer returned by Lightning. gradient_clip_valNgradient_clip_algorithmrrr) r6rfr4 clip_max_normr~rrprDclip_gradientsryr_step_lr_schedulerr9)rNrtrainer_gradient_clip_valrrs rRrz!RFDETRModelModule._step_optimizervs %,DLL:Mt$T! $ , $ 1 1 ? ?  1C< @ 9  $ ")$,,8QSW"X137&* #  (->-B   "3(?    !+/(rSc |j}|yt|tr|n|g}|D]}|j y#ttf$rYywxYw)z9Step Lightning's scheduler object when one is configured.N) lr_schedulersAttributeErrorrr~rry)rN scheduler schedulersscheduler_items rRrz$RFDETRModelModule._step_lr_schedulersf **,I   ",Y"=YI; ( "N    ! "  -   sAAAc |Dcgc]K}|jDcic]-\}}|tj|r|jn|/c}}Mc}}}Scc}}wcc}}}w)aDetach postprocessed result tensors before handing them to callbacks. Args: results: Per-image postprocessed prediction dictionaries. Returns: Per-image dictionaries with tensor values detached from the graph. )rrI is_tensorr)rresultrrs rRrz!RFDETRModelModule._detach_resultss^"  Y_XdXdXf g*#uSEOOE$:%,,. E g  g sA"2AA"A"c |jd|dddd|tjD]1\}}|j|}||j||dddd|3y)a!Log compact per-step convergence metrics for the progress bar only. Args: loss: Unscaled aggregate training loss. loss_dict: Raw criterion loss dictionary. batch_size: Current batch size used by Lightning for metric reduction metadata. rTF)rr>rrrN)rr&rget)rNrrr loss_name progress_namers rRrz-RFDETRModelModule._log_train_progress_metricss   !  )E(J(J(L  $I}MM),E} HH%   rSc|j|jDcic] \}}d|| c}}ddd||jd|ddd|ycc}}w)aLog aggregate and component validation losses. Args: loss: Aggregate weighted validation loss. loss_dict: Raw criterion loss dictionary. batch_size: Current batch size used by Lightning for metric reduction metadata. zval/FTrzval/loss)rrrrN)rrr)rNrrrrrs rR_log_val_loss_metricsz'RFDETRModelModule._log_val_loss_metricssd '0'8 9tq!tA3Z] 9!   TD44\fg :sA c |\}}|j|}|jjrZ|j|| |jj t fd D}|j | t|tj|Dcgc]}|d c}}|j||} | |dScc}w)aRun forward pass and postprocess for one validation step. Returns raw results and targets so ``COCOEvalCallback`` can accumulate them across the epoch via ``on_validation_batch_end``. Args: batch: Tuple of (NestedTensor samples, list of target dicts). batch_idx: Batch index within the validation epoch. Returns: Dict with ``results`` (postprocessed predictions) and ``targets``. c3>K|]}|vs||zywrrrs rRrz4RFDETRModelModule.validation_step..&\1P[K[y|k!n4\rrrrr) r;r4compute_val_lossrArrrrrIrrB rNrrrsrvrrrrrrrrs @@rRvalidation_stepz!RFDETRModelModule.validation_steps!**W%    - -w8I..44K\i\\D  & &tY3w< & P[['!BQ!K.!BC ""7J7"w77"Cs Cc|jjxratjj xrAtjj xr!t |jjdvS)aReturn whether fused AdamW should be used for the current training configuration. Fused AdamW is only safe when the trainer's actual precision is a BF16 variant. Checking GPU capability alone (``is_bf16_supported()``) is insufficient: on Ampere+ hardware that flag is always ``True`` even when the trainer is configured for ``32-true``, which causes a ``params, grads, exp_avgs, and exp_avg_sqs must have same dtype, device, and layout`` crash in DDP because gradient bucket views have non-matching strides in FP32. Returns: ``True`` when fused AdamW is both requested and safe to use. Examples: >>> from unittest.mock import patch >>> module = RFDETRModelModule.__new__(RFDETRModelModule) >>> module.model_config = type("Cfg", (), {"fused_optimizer": True})() >>> with patch("torch.cuda.is_available", return_value=False): ... module._use_fused_optimizer False > bf16-true bf16-mixedbf16) r3fused_optimizerrIr/ is_availableis_bf16_supportedrDrf precisionrXs rR_use_fused_optimizerz&RFDETRModelModule._use_fused_optimizersd,    - - S '') S ,,. SDLL**+/RR  rSc( |j t|j }t|jd|j}t ||}|Dcgc]}|dj s|}}tjj| j j|j}tdt j}t|j j"}|j$r"tdt'j(||z n| td j*z t j,z d fd }tjj.j1||} || ddd Scc}w) akBuild AdamW optimizer with layer-wise LR decay and LambdaLR scheduler. Uses ``trainer.estimated_stepping_batches`` for total step count so cosine annealing covers the full training run regardless of dataset size or accumulation settings. Returns: PTL optimizer config dict with optimizer and step-interval scheduler. _orig_modparams)r weight_decayfusedr_cv|kr!t|ttdz Sjdk(rpt|z ttdz z }jdjz dzdt j tj |zzzzS|jzkryy)Nr_cosineg?g?g?)rpr lr_scheduler lr_min_factorrcospilr_drop) current_stepprogresssteps_per_epochrt total_steps warmup_stepss rR lr_lambdaz9RFDETRModelModule.configure_optimizers..lr_lambdaKsl*\*U3q,3G-HHH(*  !<=c![[gMgFh@ii''1r/?/?+?3*F!dhhW[W^W^aiWiNjJj*kkkbjj?::rS)r!ry)rinterval)rr)rrreturnrp)r4r r3r6r;r requires_gradrIoptimAdamWrrrrrrrfestimated_stepping_batchesr7rceilepochs warmup_epochsrLambdaLR)rNnsmodel_for_params param_dictsprr microbatchesr!rrrtrr s @@@@rRconfigure_optimizersz&RFDETRModelModule.configure_optimizerssj   $T%6%6 ; #4::{DJJG$R)9: "-KQ81J1JqK KKK%% uu++ & q#b&9&9":;4<<BBC CGB_B_C499\,<<= >eq a !9:?R-=-==>   KK,,55i95U #*3H  ULs F-Fc|jrB|r?|dkDr9tjjj |j |yyyt ||||y)aOverride PTL gradient clipping to support fused AdamW. PTL's AMP precision plugin refuses to clip gradients when the optimizer declares it handles unscaling internally (fused=True). When fused is active we are on BF16 (no GradScaler) so ``clip_grad_norm_`` is correct. For the non-fused path (FP16 + GradScaler or FP32) we delegate to ``super()`` to preserve scaler-aware unscaling. Args: optimizer: The current optimizer. gradient_clip_val: Maximum gradient norm. gradient_clip_algorithm: Clipping algorithm; forwarded to super() for the non-fused path. rrN)rrInnutilsclip_grad_norm_rr1r)rNrrrrQs rRrz RFDETRModelModule.clip_gradients]s^$  $ $ %6%:..t/@BST&;  G ""3(? # rSc |\}}|j|}|jjr[|j|| |jj t fd D}|j d|dt|tj|Dcgc]}|d c}}|j||} | |dScc}w)a*Run forward pass and postprocess for one test step. Mirrors :meth:`validation_step` so ``COCOEvalCallback`` can accumulate results via ``on_test_batch_end`` when ``trainer.test()`` is called (e.g. from :class:`~rfdetr.training.callbacks.BestModelCallback` at end of training). Args: batch: Tuple of (NestedTensor samples, list of target dicts). batch_idx: Batch index within the test epoch. Returns: Dict with ``results`` (postprocessed predictions) and ``targets``. c3>K|]}|vs||zywrrrs rRrz.RFDETRModelModule.test_step..rrz test/lossT)rrrr) r;r4compute_test_lossrArrrrrIrrBrs @@rR test_stepzRFDETRModelModule.test_stepys!**W%    . .w8I..44K\i\\D HH[$$3w j \%CDNwDWD]D]D_'`DAq1 a'`J| $ : % +<(!!::  #j 0-78O-PD * MMB   1(asB c:|jj|y)zReinitialize the detection head for a new class count. Args: num_classes: New number of classes (excluding background). N)r;reinitialize_detection_head)rN num_classess rRrIz-RFDETRModelModule.reinitialize_detection_heads ..{;rS)r3r r4r r#None)r#rK)rrrrsrr#rK)rrrrsrr#ztorch.Tensor | dict[str, Any])rdict[str, torch.Tensor]rlist[dict[str, torch.Tensor]]r#z:tuple[dict[str, torch.Tensor], torch.Tensor, torch.Tensor])r torch.TensorrrNr#rN)rzrNr#rK)rsrr#r5)rtorch.optim.Optimizerr#rK)rrMr#rM)rrNrrLrrr#rK)rrrrsrr#Dict[str, Any])r#r5)r#rP)NN)rrOrzOptional[float]rz Optional[str]r#rK)r)rrrrsrr;rr#r)rFzdict[str, Any]r#rK)rJrr#rK)r __module__ __qualname____doc__r2rYr{rrrrrrrr staticmethodrrrrpropertyrr1rr9r<rGrI __classcell__)rQs@rRr(r(*sW?AJU206\^|#/(#//#/ D #/J11!1  1,h >06 "    ##+#  #  #Jhh+h  h  h.82  6> F.215 (+"/   8845"6pr&__annotations__r(rrSrRrcs ;" -- >52;Udd7. ! (&" 0n x <x