^jsyZdZddlmZddlmZmZmZmZmZm Z ddl Z ddl Z ddl m Z ddl mZddlmZddlmZmZdd lmZdd lmZdd lmZdd lmZdd lmZeZdZerddl m!Z!de"fdZ#Gdde jHjJjLZ'de(de(fdZ)Gdde Z*y)zALightningDataModule for RF-DETR dataset construction and loaders.)Path) TYPE_CHECKINGAnyListLiteralOptionalTupleN)LightningDataModule) DataLoader)_namespace_from_configs) ModelConfig TrainConfig) build_dataset) AUG_CONFIG)box_xyxy_to_cxcywh) get_logger)make_collate_fn)FigurereturncBddlm}t|jdS)zReturn ``True`` when the runtime has a CUDA accelerator available. Uses the fork-safe global ``DEVICE`` constant instead of direct ``torch.cuda.is_available()`` calls to avoid creating a CUDA context in fork-based notebook/DDP workflows. rDEVICEcuda) rfdetr.configrstr startswithrs f/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/rfdetr/training/module_data.py_has_cuda_devicer s % v; ! !& ))c eZdZdZ d dej j jdededdfdZ defdZ d ede fd Z y) GradAccumAlignedDatasetuDataset wrapper that pads length to a multiple of ``effective_batch_size * world_size``. Workaround for https://github.com/Lightning-AI/pytorch-lightning/issues/19987: PTL fires the optimizer on partial accumulation windows at the tail of the dataset, causing the last optimizer step to be under-scaled. Padding the dataset to a multiple of ``effective_batch_size * world_size`` ensures that ``drop_last=True`` on the DataLoader becomes a true no-op — every accumulation window is always complete. Padding indices are drawn randomly from the original dataset. Because RF-DETR uses online augmentation, each padded sample receives a fresh random augmentation at ``__getitem__`` time, so it behaves like a new training example rather than a true duplicate. This wrapper can be removed once the upstream PTL issue is resolved. Args: dataset: The underlying dataset to wrap. effective_batch_size: ``batch_size * grad_accum_steps``. world_size: Number of DDP processes (default 1 for single-GPU/CPU). The alignment unit is ``effective_batch_size * world_size`` so that after PTL's ``DistributedSampler`` splits samples across ranks each rank still receives an exact multiple of ``effective_batch_size``. dataseteffective_batch_size world_sizerNc|dkrtd||dkrtd|||_t||_||z}|j|z}||z |z}t j }|j d|dkDr2t jd|j|f|jng|_ |j|z|_ y)Nz'effective_batch_size must be >= 1, got zworld_size must be >= 1, got r) generator) ValueError_datasetlen_dataset_lengthtorch Generator manual_seedrandinttolist _pad_indices_length)selfr#r$r%pad_unit remainder pad_countpad_index_generators r__init__z GradAccumAlignedDataset.__init__As ! #FG[F\]^ ^ >!4#7#77cT=N=NsUYUiUiOi=j }}[))r )r') __name__ __module__ __qualname____doc__r-utilsdataDatasetintr9r<rr@r rr"r"+sf2 8!!))8"8 8  8<*s*s*r r"backendcP|dk7r|Stsy ddl}y#t$rYywxYw)aResolve ``"auto"`` to ``"cpu"`` or ``"gpu"`` based on runtime availability. For ``"cpu"`` and ``"gpu"`` the value is returned unchanged. For ``"auto"`` the function checks CUDA and kornia availability and returns ``"gpu"`` only when both are present; otherwise ``"cpu"``. Called before dataset construction so that ``gpu_postprocess`` in the dataset builders always matches what the DataModule will actually do in ``on_after_batch_transfer``. Args: backend: Value of ``TrainConfig.augmentation_backend``. Returns: Resolved backend string, either ``"cpu"`` or ``"gpu"``. Examples: >>> _resolve_augmentation_backend("cpu") 'cpu' >>> _resolve_augmentation_backend("gpu") 'gpu' autocpurNgpu)rkornia.augmentation ImportError)rJkornias r_resolve_augmentation_backendrRks7*&  " s  %%c eZdZdZdededdffd ZdeddfdZde fd Z de fd Z de fd Z de fd Z d$d dddededdedeeefdzddf dZed%dededefdZdeddej,j.j0dzfdZedej,j.j0dededzfdZd&dZdededefd Zede e!efd!Z"ded"ejFdedefd#Z$xZ%S)'RFDETRDataModulea LightningDataModule wrapping RF-DETR dataset construction and data loading. Args: model_config: Architecture configuration (used for resolution, patch_size, etc.). train_config: Training hyperparameter configuration (used for dataset params). model_config train_configrNc t|||_||_|j|j z}|dkr)t d|d|jd|j dt||_d|_ d|_ d|_ d|_ d|_ d|_|jj|_ddlm}t'|jj(j+}|d v}|jj, |d k(xr!|nt/|jj,|_|jj2|j dkDnt/|jj2|_|j dkDr:|jj6|jj6|_yd |_yd|_y) Nrz-Computed collate block_size must be > 0, got z from patch_size=z and num_windows=.) block_sizeFr>rNrLrr)superr9rUrV patch_size num_windowsr)r _collate_fn_dataset_train _dataset_val _dataset_test_kornia_pipeline_kornia_normalize_kornia_setup_done num_workers _num_workersrrr acceleratorlower pin_memorybool _pin_memorypersistent_workers_persistent_workersprefetch_factor_prefetch_factor)r4rUrVrYrrguses_cuda_accelerator __class__s rr9zRFDETRDataModule.__init__s (( ",,|/G/GG ??,/ 0G0G/HI##/#;#;"!> )$++778>>@ +/F F  ++3v  7"7d''223   33;    !d''::;    q 595F5F5V5V5b!!11  !hi  !%)D !r stagec&|jj}t|j|j}|dk(rt |jj }||jj k7r||_|jj r|dk7rtd|d|jtd|||_|jtd|||_ |js|jd|_ yy|d k(r |jtd|||_ yy|d k(r=|j0|jjd k(rd nd}t||||_ yy|d k(r |jtd|||_ yyy) uBuild datasets for the requested stage. PTL calls this on every process before the corresponding dataloader method. Datasets are built lazily — a dataset is only constructed once even if ``setup`` is called multiple times. Args: stage: PTL stage identifier — one of ``"fit"``, ``"validate"``, ``"test"``, or ``"predict"``. fitrMzGPU augmentation backend 'zi' does not support keypoint transforms. Set augmentation_backend='cpu' when use_grouppose_keypoints=True.NtrainvalTvalidatetestroboflowpredict)rU resolutionr rVrRaugmentation_backenduse_grouppose_keypointsr)r_rr`rd_setup_kornia_pipelinera dataset_file)r4rrr{nsresolvedsplits rsetupzRFDETRDataModule.setups&&11 $T%6%68I8I J E> 5T5F5F5[5[\H4,,AAA*2'  88X=N 0 ;XX""*&3GR&L#  ($1%Z$H!**++-*.'+j   ($1%Z$H!) f_!!)"&"3"3"@"@J"NTY%25"j%I"*i   ($1%Z$H!) r c |j}|jj}||jjz}|j}t ||t zkrtjdt ||t ztjjj|d|t z}t||||j||j|j |j"S|j$rt'|j$ddnd}t)|||}t||dd|j||j|j |j" S)aReturn the training DataLoader. Uses a replacement sampler when the dataset is too small to fill ``_MIN_TRAIN_BATCHES`` effective batches (matching legacy behaviour in ``main.py``). Otherwise wraps the dataset with :class:`GradAccumAlignedDataset` to ensure its length is an exact multiple of ``effective_batch_size * world_size`` (workaround for https://github.com/Lightning-AI/pytorch-lightning/issues/19987) and then uses ``shuffle=True, drop_last=True`` so that PTL can auto-inject ``DistributedSampler`` in DDP mode. Returns: DataLoader for the training dataset. zCTraining with uniform sampler because dataset is too small: %d < %dT) replacement num_samples) batch_sizesampler collate_fnrerirlrnr%r')rshuffle drop_lastrrerirlrn)r_rVrgrad_accum_stepsrfr+_MIN_TRAIN_BATCHESloggerinfor-rErF RandomSamplerr r^rkrmrotrainergetattrr")r4r#rr$rerr%s rtrain_dataloaderz!RFDETRDataModule.train_dataloadersQ%%&&11 )D,=,=,N,NN'' w<.1CC C KKUG $'99  kk&&44 03EE5G %++'++#'#;#; $ 5 5  EILL'$,, a@VW )'3GT !''#''#77 11  r c ,t|j|jjtj j j|jd|j|j|j|j|j S)zReturn the validation DataLoader. Returns: DataLoader for the validation dataset with sequential sampling. Frrrrrerirlrn r r`rVrr-rErFSequentialSamplerr^rfrkrmror;s rval_dataloaderzRFDETRDataModule.val_dataloader:w    ((33KK$$66t7H7HI''))''#77 11  r c ,t|j|jjtj j j|jd|j|j|j|j|j S)z|Return the test DataLoader. Returns: DataLoader for the test dataset with sequential sampling. Fr) r rarVrr-rErFrr^rfrkrmror;s rtest_dataloaderz RFDETRDataModule.test_dataloaderLsw    ((33KK$$66t7I7IJ''))''#77 11  r c ,t|j|jjtj j j|jd|j|j|j|j|j S)zReturn the predict DataLoader (reuses the validation dataset, no augmentation). Returns: DataLoader for the validation dataset with sequential sampling. Frrr;s rpredict_dataloaderz#RFDETRDataModule.predict_dataloader^rr )columns figure_sizecountr)rurvrxrrrc  ddlm}ddl}ddl}ddlm}ddlm} |dkrtd|d|dkrtd|d|}|jAd}tC|tDjFrtI|dtI|d }}n0tI|jJdtI|jJd}}|jAdtEjLdtDjN}|jAdtEjLd tDjP}|jSdkDrtEjT||||gtDjN}| |j3j5|z} |j3j5jj=tH}!|jAd!}"|"?|"jSdkDr,|"j3j5jnd}#|jW| jj=|jN|!|#"}$|!D%cgc]"}%||%t|kr||%n tY|%$}&}%|#!|j[j]||$#}|j_d $j]||$#}|jad%d&j]||$|&'}|jAd(}'|'|'jSdkDr|'j3j5jj=|jN}(|(d)ddf|j#||g|jNz})|(d*dkD}*|jc|)|*j=|jN|j3j5jj=tH|*d+|*i,}+|jed-.j]||+/}|jg|| |jhn|d0|d1},|jk|jm|,d23|j)d|jo|S#t $r} t d| d} ~ wwxYwcc}%w)4aBuild a private diagnostic figure for transformed dataset samples. Samples the dataset after RF-DETR dataset transforms, so boxes and keypoints match the model input tensors rather than raw annotation JSON. Args: count: Maximum number of samples to render. split: Dataset split to visualize. columns: Number of subplot columns. figure_size: Optional Matplotlib figure size ``(width, height)`` in inches. When omitted, the size is derived from the grid shape. Returns: Matplotlib figure containing the annotated sample grid. When the dataset includes instance masks, they are rendered as coloured overlays before bounding boxes and labels. Raises: ValueError: If ``count`` or ``columns`` is not positive. Example: >>> # dm = RFDETRDataModule(model_config, train_config) >>> # figure = dm._show_samples(3, split="train") rNzwRFDETRDataModule._show_samples() requires visualization dependencies. Install them with `pip install 'rfdetr[visual]'`.)box_cxcywh_to_xyxyzcount must be positive, got rXzcolumns must be positive, got rZz)figure_size must contain two values, got r'z)figure_size values must be positive, got zCould not build dataset split z for visualization.)g:ܟwg$I$IggE#)g!:ܟw@gm۶m@grq@)meanstdr)figsize)dtypeoffF)strictgg?sizeboxes)rlabels)rmasks)xyxyclass_idmask)scene detections) thicknessg?) text_scale text_padding)rrr keypoints.).rZvisible)xykeypoint_confidencerrrFr)radius)r key_points[] )fontsize)8matplotlib.pyplotpyplotnumpy supervisiontorchvision.transforms transformsrPrfdetr.utilities.box_opsrr)r+_get_dataset_for_visualization RuntimeError Normalizemaxminsubplotsasarrayobjectreshapeaxis class_namesziprange_source_image_pathdetachrMascontiguousarrayclip transposeastypeuint8get isinstancer-TensorrHshapezerosfloat32int64numeltensor Detectionsr MaskAnnotatorannotate BoxAnnotatorLabelAnnotator KeyPointsVertexAnnotatorimshowname set_title_ellipsize_sample_title tight_layout)-r4rrrrpltnpsvTerrrr# inv_normalizerowsfigureaxes axes_arrayrr sample_index image_tensortarget image_pathimage image_arrayrrheightwidthrrscaler class_ids masks_tensor mask_arrayrr labels_textrkeypoints_array keypoint_xykeypoint_visibilityrtitles- r _show_sampleszRFDETRDataModule._show_samplesps-@  +  $ . @ A:;E7!DE E a<=gYaHI I  ";1$ #L[MYZ![\\1~"k!n&9 #L[MYZ![\\55e< ?!?yH[\] ] A1$ 1s5#g,/'9A='IJ||D';;a1w;XY\`X`Ja|b ZZFZ3;;B?  D IIe  && "%j%E3w<8P2QZ_"`8  D,#*<#8 L&00,GJ!,/E,,.,,.446K(("''+2G2G1a2PRUWZ*[^a*a)i)ijljrjr)stE::f%D$ - #DG c$q'l #L$6$6r$: ;SASASTVAW=XJJw F%--(PQEZZ%++d%++*NOF{{}q  eVUF%C5==Y)%,,.*<*<*>?%G"MMO//1779@@E %zz'2 $/L4F4F4H14L!'')--/557  ]] 0C0CBJJ0OZcjt]u %. .9-DTWXcTdIdK)jmnvjww ),,.77ePZ7[E!4==EV`=a))Sq)IRR)&S  ;/I$):Q)>"+"2"2"4"8"8":"@"@"B"I"I"**"U-c2A2g6UFO[][e[e9ff &5f&=&A#\\"(;(B(B2::(N#]]_00288:AA#F/#%89 * **!*4==EV`=a KK '1'=JOOeWAl^[\C]E NN477>N L IIe q8 t  y D  vsY(6'Z( Z1 Y==Zr max_lengthct||kr|S|dkrd|zStd|dz dz}|dz |z }|d|d|| dS)aShorten long sample titles so subplot grids do not overflow. Args: title: Raw title text, usually an image file name. max_length: Maximum returned character count including the ellipsis. Returns: Original title when it already fits, otherwise a middle-ellipsized string that preserves the start and file suffix. rrXr'rZNz...)r+r)rr keep_left keep_rights rrz(RFDETRDataModule._ellipsize_sample_titlesm u: #L ?# #JNq01 !^i/  #$Czkl(;'<==r c|dk(r|jd|jS|dk(r|jd|jS|dk(r|jd|jSt d|d)z7Return a built dataset split for private visualization.rurtrvrwrxzUnsupported split rX)rr_r`rar))r4rs rrz/RFDETRDataModule._get_dataset_for_visualizationsw G  JJu && & E> JJz "$$ $ F? JJv %% %-eYa899r r#rct|dd}t|dd}t|dd}|||y||}|j|d}t||dz S)z:Return a source image path for common COCO-style datasets.rootNidscocor file_name)rloadImgsr)r#r image_folder image_idsrimage_id image_infos rrz#RFDETRDataModule._source_image_pathsrw5 GUD1 w-  9#4 \*]]8,Q/ L!J{$;;;r cv|jj}|dk(ry|dk(r&tstj dy ddl}n|dk(rts td ddl}dd lm }m }||jj|jjnt|jj|jj |_||_tj%d |y#t $rtj dYywxYw#t $r}t d |d}~wwxYw) aXResolve augmentation backend and build the Kornia pipeline if applicable. Called once during ``setup("fit")``. When ``augmentation_backend`` is ``"cpu"`` this is a no-op. For ``"auto"`` the method falls back silently when CUDA or Kornia are unavailable. For ``"gpu"`` missing requirements raise hard errors. rMNrLzFaugmentation_backend='auto': no CUDA, falling back to CPU augmentationrzIaugmentation_backend='auto': kornia not installed, using CPU augmentationrNz1augmentation_backend='gpu' requires a CUDA devicezLGPU augmentation requires kornia. Install with: pip install 'rfdetr[kornia]')build_kornia_pipelinebuild_normalize) with_masksz3Kornia GPU augmentation pipeline built (backend=%s))rVr|rrwarningrOrPr!rfdetr.datasets.kornia_transformsr r! aug_configrrUr{segmentation_headrbrcr)r4rJrQrr r!s rr~z'RFDETRDataModule._setup_kornia_pipeline#s##88 e   f #%gh * #%"#VWW * ] 5,0,=,=,H,H,TD   ( (Zd    ( (((::!  "1!2 I7S+ jk  !b s*C:#D:DD D8' D33D8batchdataloader_idxc|j"|jjr |j|Sddlm}m}m}ddlm}|\}}|j} |jj| j|jj| j||| j\} } |jjrz| jdd\} } ||| j| jd| | }|j| | |\}}}|j|}||| |g|jddd|i}n>|j| | \}}|j|}||| |g|jdd}|jdd\}}|D]A}|d }|j!dk(r|j#||||g}t%||z |d <C|||j&|f}|S) aApply Kornia GPU augmentation after the batch is transferred to device. When ``_kornia_pipeline`` is set and the trainer is in training mode, augmentation and normalization are applied on the GPU. Validation and test batches pass through unchanged. Segmentation models use a mask-aware pipeline (``with_masks=True``) so images, boxes, and per-instance masks are augmented in sync. Args: batch: Tuple of ``(NestedTensor, list[dict])`` already on device. dataloader_idx: Index of the current dataloader. Returns: The (possibly augmented) batch. Nr) collate_boxes collate_masks unpack_boxes) NestedTensorrr')n_max image_height image_width masks_augr)rtrainingrbr$r*r+r,rfdetr.utilities.tensorsr-tensorstodevicercrUr&rr new_tensorrr)r4r'r(r*r+r,r-samplestargetsimg boxes_paddedvalidr/r0 masks_paddedimg_aug boxes_augr1rrrrrs ron_after_batch_transferz(RFDETRDataModule.on_after_batch_transferKs << t||'<'<@U@U@]L``9 oo   , !!#**-+GSZZ@ e    . .(+ "# %L+(5;;q> bmL-1,A,A#|Ua,b )GY ,,W5G"9eWgw}}RS?Qg]fgG!%!6!6sL!I GY,,W5G"9eWRw}}RS?QRG bc*  @F7OE{{}!$$eVUF%CDE07%?F7O  @ gw||4g> r c&|j|jfD]}|t|dd}|t|ds#t|dd}| t|dd}t |t ra|r_t |}dg|dzz}t|jD]+\}}|jj|}|$|d||<-|cSt|jjD cgc]} |j| dc} cSycc} w)aaClass names from the training or validation dataset annotation file. Reads category names from the first available COCO-style dataset. Returns ``None`` if no dataset has been set up yet or the dataset does not expose COCO-style category information. Returns: Sorted list of class name strings, or ``None``. Nrcats label2catr'r) r_r`rhasattrrdictrsorteditemsrBrkeys) r4r#rrC max_labelnameslabel category_idcategoryks rrzRFDETRDataModule.class_namess++T->->? PG7FD1DGD&$9#G[$? $ 'k4 @Ii.9 #IIDIM2E.4Y__5F.G<*{#'99==#=#/+3F+;E%L<!L6>UggVgsA8A2$A82A8)ru)0)rN)&rArBrCrDr rr9rrr rrrrrHrtuplefloatr staticmethodrr-rErFrGrrrr~r r@propertyrrrr6rU __classcell__)rqs@rrTrTs7)[7) 7)PT7)z-I3-I4-I^8 *8 t  $  $ J *29A 26 AA-.A  A 5%<(4/ A AF>s>>S>>&:-.:    ! !D (:  res HEE 1'52)27.4 (*$*=*ekk..66=*@33Bg *g r