^jq dZddlZddlmZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZmZmZmZmZmZdd lmZmZdd lmZmZmZGd dedZ d$ddde!e"e"fdzde"dzfdZ#de!e$e"fde%e"e"ffdZ&de'ejPde'e'e"de%e'ejPe'e'e"ffdZ)dZ*d%dZ+ d&de,de,d e%e"e"fdzfd!Z-eGd"d#e Z.d#gZ/y)'zImage processor class for EoMT.N)Union) functional)TorchvisionBackend)'SemanticSegmentationPostProcessorOutput) BatchFeature)get_size_with_aspect_ratiogroup_images_by_shapereorder_images)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDChannelDimension ImageInputPILImageResamplingSizeDict) ImagesKwargsUnpack) TensorTypeauto_docstringfilter_out_non_signature_kwargsc,eZdZUdZeed<edzed<y)EomtImageProcessorKwargsa{ do_split_image (`bool`, *optional*, defaults to `self.do_split_image`): Whether to split the input images into overlapping patches for semantic segmentation. If set to `True`, the input images will be split into patches of size `size["shortest_edge"]` with an overlap between patches. Otherwise, the input images will be padded to the target size. ignore_index (`int`, *optional*, defaults to `self.ignore_index`): Label to be assigned to background pixels in segmentation maps. If provided, segmentation map pixels denoted with 0 (background) will be replaced with `ignore_index`. do_split_imageN ignore_index)__name__ __module__ __qualname____doc__bool__annotations__inty/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/eomt/image_processing_eomt.pyrr*s*r#rF)totalsegmentation_map torch.Tensorinstance_id_to_semantic_idrcr|tj|dk(||dz }tj|}||||k7}|Dcgc]}||k( }}|rtj|d}n/tjdg|j |j }|vtj|j d|j }t|D]9\}}|||jdzn|j}||dz n|||<;n|}|j|jfScc}w)Nrdimdevice) torchwhereuniquestackzerosshaper. enumerateitemfloatlong) r&r(r all_labelsi binary_maskslabelslabelclass_ids r$-convert_segmentation_map_to_binary_masks_fastr?:sE  ;;'71'(>#?HXH_H_` "-Z--a09I9P9PQ!*- OHAu1|G_5::z-reorder_patches_and_offsets..gs!Qr#)key)listzipsort)rHrIcombinedsorted_offsetssorted_patchess r$reorder_patches_and_offsetsrUasG C)*H MM'M(%((^"NN  n!5 55r#c|jd|jdcxk(r|jdk(stdtd|j|||kDz}||||||fS)a  Binarize the given masks using `object_mask_threshold`, it returns the associated values of `masks`, `scores` and `labels`. Args: masks (`torch.Tensor`): A tensor of shape `(num_queries, height, width)`. scores (`torch.Tensor`): A tensor of shape `(num_queries)`. labels (`torch.Tensor`): A tensor of shape `(num_queries)`. object_mask_threshold (`float`): A number between 0 and 1 used to binarize the masks. Raises: `ValueError`: Raised when the first dimension doesn't match in all input tensors. Returns: `tuple[`torch.Tensor`, `torch.Tensor`, `torch.Tensor`]`: The `masks`, `scores` and `labels` without the region < `object_mask_threshold`. rz1mask, scores and labels must have the same shape!)r4 ValueErrorne)masksscoresr<object_mask_threshold num_labelsto_keeps r$remove_low_and_no_objectsr^msy( KKNfll1o @a @LMM ALMMii #v0E'EFG >6'?F7O ;;r#c||k(}|j}|||k\}|j}||z} | j} |dkDxr |dkDxr| dkD} | r||z } | j|kDsd} | | fS)NrF)sumr6) mask_labels mask_probskmask_thresholdoverlap_mask_area_thresholdmask_k mask_k_area original_mask original_area final_maskfinal_mask_area mask_exists area_ratios r$check_segment_validityrns A F**,KqM^3M!%%'M-'J nn&O/Oma&7OOa>K  ""r#rdre target_sizecb||jdn|d}||jdn|d}tj||ftj|jdz } g} |j }|ddddf|zj d} d} i} t|jdD]~}||j}t| ||||\}}|s+|r||vr|| vr | || |<>| | |<| | |<t||jd}| j| ||d| dz } | | fS)Nr*r)dtyper.idlabel_idscore) r4r/r3r8r.sigmoidargmaxranger6rnroundappend)rb pred_scores pred_labels stuff_classesrdreroheightwidth segmentationsegmentsracurrent_segment_idstuff_memory_listrc pred_classrlrj segment_scores r$compute_segmentsrs%0$7Z  a [^F#.#6J  Q KNE;;ejjIZIZ[^__LH##%Jq$}- :BB1EK(* ;$$Q' (  ^((* #9 Q8S#  Z  Z=8..+ ALqM[1_=VW{# MAZ*;;<=E*$C~$Q59a%78 $Q1eCi%78 %.u||KQ/O%P M! 6v&$$mI&>s%KL M M %%r#c0|j\}}}}t|j|jxs |jd\}}t d||z }t d||z } d| d|f} t j jj|| dd} | S)z5Pads the image to the target size using zero padding.rrconstantg)modevalue) r4rGrCrDrr/nnrpad) rrrrrrrErFpad_hpad_wpadding padded_imagess r$_padzEomtImageProcessor._pads$ll1fe&5"00$BSBSBiW[WiWi j' # |A}v-.A|e+,eQ&++//jX[/\ r#segmentation_mapsr(c (t||||fi|S)a' segmentation_maps (`ImageInput`, *optional*): The segmentation maps to preprocess for corresponding images. instance_id_to_semantic_id (`list[dict[int, int]]` or `dict[int, int]`, *optional*): A mapping between object instance ids and class ids. )r preprocess)rrrr(rrs r$rzEomtImageProcessor.preprocesss!w!&*;=Wb[abbr#do_convert_rgbinput_data_formatreturn_tensorsr.z torch.devicec f|j||||}|jdd} |j} i} |j|fi| \} } | | d<| | d<||j|ddtj }|j}|j ddtjd |jdd |i|\}}|Dcgc]0}|jd jtj2}}gg}}t|D]P\}}t|tr||}n|}t!||| \}}|j#||j#|R|| d <|| d<| r&| Dcgc]}tj$|c}| d<t'| |gdScc}wcc}w)z/ Preprocess image-like inputs. )rrrr.rN pixel_valuesrrqF)rexpected_ndimsrr) do_normalize do_rescaleresamplerr)rra class_labels)rrar)data tensor_typeskip_tensor_conversionr")_prepare_image_like_inputspopcopy _preprocessrFIRSTupdaterNEARESTsqueezetor/int64r5 isinstancerOr?r|tensorr)rrrr(rrrr.rr images_kwargsrprocessed_imagesrprocessed_segmentation_mapssegmentation_maps_kwargsrr&raridx instance_idrYclassesrIs r$_preprocess_image_like_inputsz0EomtImageProcessor._preprocess_image_like_inputs!s00.L]fl1 zz.$7  *:$*:*:6*S]*S'-/^ -_  (*.*I*I( $"2"8"8 +J+ '(.{{} $ $ + +$)"' 2 : :  .>T-=-=.2.6N. * 'Up+@P ((+..u{{;+ '+)+BK)23N)O -%%8$?"!%>W_!`,: "5 ) ; 68LM 3Hbr3s 0N0%']G)7)=)=)? .%~EYE_E_Ea$$%A\>IqenrwewL$ $*.):):>4Qa)b& w}-$$W-  .%@$W !FM 3Hbr3s 0N0TbThThTj;P5.tyy66N$N4HIF/DV^n/o,,#% %3%9%9%; = !E>!77 NL*V_N/= $U +  = **BDXY..1$sF %F F#segmentation_logitsr target_sizescT|jd}g}g}|D]~}t||d|d\} } |jtj|| | f|j |jtj|| | f|j t |D]\} \} } }|| d|| dkDr6|| dd| |ddfxx|| z cc<|| dd| |ddfxxdz cc<Q|| dddd| |fxx|| z cc<|| dddd| |fxxdz cc<g}t t||D]d\}\}}||jdz }tjjj|d||d d d}|j|f|S) a Reconstructs full-size semantic segmentation logits from patch predictions. Args: segmentation_logits (`torch.Tensor`): A tensor of shape `(num_patches, num_classes, patch_height, patch_width)` representing predicted logits for each image patch. patch_offsets (`list[tuple[int, int, int]]`): A list of tuples where each tuple contains: - `image_index` (int): Index of the original image this patch belongs to. - `start` (int): Start pixel index of the patch along the long dimension (height or width). - `end` (int): End pixel index of the patch along the long dimension. target_sizes (`list[tuple[int, int]]`): list of original (height, width) dimensions for each image before preprocessing. size (`dict[str, int]`): A size dict which was used to resize. r*rCrDr-rN)minN.bilinearFrr align_corners) r4r r|r/r3r.r5rPclamprr interpolate)rrrrr num_classesaggregated_logits patch_counts image_sizerr patch_idx image_idx patch_start patch_endreconstructed_logitsr logit_sumcountaveraged_logitsresized_logitss r$merge_image_patchesz&EomtImageProcessor.merge_image_patchess 0*//2  & nJ6z4CXZ^_mZnoMFE  $ $U[[+vu1MViVpVp%q r    [&%,HQdQkQk l m n ?H >V J :I: ; I&q)L,CA,FF!),Q I0Eq-HIM`ajMkkIY';y+@!(CDID!),Q;y3H-HIM`ajMkkIY'1k).C(CDID  J "'05F 1U'V 8 #C#)U'%++!+*<6J+ 'M<15a-,6VWN$xx22>>y) J^c ?     ! !"2 3 4r#return_segmentation_scoreszBlist[torch.Tensor] | list[SemanticSegmentationPostProcessorOutput]c||n |j}|j}|j}|j}t |}t j jj||d}|jddddf} |j} t jd| | } |r|j| |||} nrg} tt| D]Y} t j jj| | jd|| dd }| j!|d[| Dcgc]!}t#|j%d|d  #}}|s|Dcgc]}|j&}}|Scc}wcc}w) a@ Converts the output of [`EomtForUniversalSegmentation`] into semantic segmentation maps. Args: outputs ([`EomtForUniversalSegmentation`]): Raw outputs of the model. target_sizes (`list[tuple[int, int]]`): A list of tuples (`tuple[int, int]`) containing the target size (height, width) of each image in the batch. size (`dict[str, int]`, *optional*): The size to which the intermediate masks are interpolated. Defaults to `self.size`. return_segmentation_scores (`bool`, *optional*, defaults to `False`): Whether to return segmentation scores alongside the segmentation map. When `True`, each element of the returned list is a [`SemanticSegmentationPostProcessorOutput`] with fields `segmentation` (class IDs, shape `(height, width)`) and `segmentation_scores` (shape `(num_classes, height, width)`). Returns: `list[torch.Tensor]` or `list[SemanticSegmentationPostProcessorOutput]`: When `return_segmentation_scores=False` (default), a list of length `batch_size` where each item is a segmentation map of shape `(height, width)` with class IDs. When `return_segmentation_scores=True`, a list of [`SemanticSegmentationPostProcessorOutput`] with fields `segmentation` (class IDs, shape `(height, width)`) and `segmentation_scores` (shape `(num_classes, height, width)`). In both cases, `(height, width)` corresponds to the target size. Nrrrr+.zbqc, bqhw -> bchwrFr)rsegmentation_scores)r)rmasks_queries_logitsclass_queries_logitsrrGr/rrrsoftmaxrxeinsumrrzlen unsqueezer|rryr)routputsrrr rrr output_size masks_classes masks_probsr output_logitsrrlogitsemantic_segmentationr6s r$"post_process_semantic_segmentationz5EomtImageProcessor.post_process_semantic_segmentations>'tTYY&;;&;;-- %d+ $xx22>>  ? -444 {[  445H-YegklMMS!456 8!&!4!4!@!@',6616=%c*#"' "A" $$^A%67 8' !  4&+llql&9RWX ! ! *CX$Y4T%6%6$Y !$Y$$! %Zs 9&E>'F thresholdrdrerc ||n |j}|j}|j} | jd} | jddz } t |} t j jj|| d}|j|||} | jdjd\}}g}t| D]}t| ||||||| \}}}|jddkrH|||n|jdd\}}t j||fdz }|j|gdyt!|||||||||nd \}}|j||d|S) zIPost-processes model outputs into final panoptic segmentation prediction.Nrr r*rr r+r segments_info)rbr}r~rrdrero)rrrr4rGr/rrrrrrrzr^r3r|r)rrrrrdrerrrr batch_sizer\rmask_probs_batchpred_scores_batchpred_labels_batchresultsr:rbr}r~rrrrs r$"post_process_panoptic_segmentationz5EomtImageProcessor.post_process_panoptic_segmentationBs'tTYY&;;&;;)//2 )//3a7 %d+ $xx22>>  ?   ++,@,PTU/C/K/KPR/K/S/W/WXZ/[,,z" VA3L #%6q%9;LQ;OQZ\f4 0J [ "a'3?3K QQ[QaQabcbdQe ${{FE?;a?  rRS%5%''+-,G/;/GLOT& "L( NNL8T U- V.r#c h||n |j}|j}|j}t|}tj j j||d}|j|||}|j} |jd} |jd} g} t| D]} || }|| }|jddddfjd\}}|dkDj}|jj!d |j!d zj#d |j!d j#d d zz }||z}t j$|| | d z }gg}}d}t| D]}||j'}t j(||dk(r2||k\s8||||d k(<|j+|||j't-|d d |d z }|j+||| j+||d| S)zDPost-processes model outputs into Instance Segmentation Predictions.Nrr rr r+.r*gư>r-rsrtr)rrrrGr/rrrrr.r4rzrrr7rxflattenr`r3r6allr|r{)rrrrrrrrr!r.r  num_queriesr$r: mask_pred mask_classrZ pred_classes pred_masks mask_scoresr}r instance_mapsrrjrws r$"post_process_instance_segmentationz5EomtImageProcessor.post_process_instance_segmentation{sa'tTYY&;;&;;%d+ $xx22>>  ?   ++,@,PTU%,,)//2 *004 z"! VA(+I-a0J$.#5#5"#5#=c3B3h#G#K#KB#O FL#a-..0J%,,.66q9J%*5!_'!+&!((A7 8 NNL8T UC! VDr#NN)N)NF)皙??r4NN)r4N)2rrrr valid_kwargsrBILINEARrr rr rrdefault_to_squarerrrrrrrrr/TensorrrOr!tuplerrrrdictrrrrstrrrrr7rrrrr%rr2 __classcell__)rs@r$rrsD+L!**H&J$I # 6DIJLN FL#(@!A#&5<<&x&PTUXPY&^cdhjndn^o&8 5<< x ELL 8<<@ c c  -4 c%)cNT$9 c 12 c  c c.59L L &,L %)cNT$9 L  L , L j(4/L c>)*T1L 12L  L \7/^$7/7/ 7/ L 7/  7/7/7/7/7/DK'$.7/4;&-7/+7/r7$"\\7$E#sC-017$5c?+ 7$ 38n 7$ ell  7$r"\\5c?+38n  ell  2'++0 J%5c?+J%38nt# J% %) J% N J%` #-0*.&*75c?+7 7  7 &+ 7Cy4'738nt#7r%& &* ?5c?+? ? 38nt# ?'?r#rr3)r5r4)r5r4N)0rrtypingrr/torchvision.transforms.v2rtvFimage_processing_backendsrimage_processing_outputsrimage_processing_utilsrimage_transformsr r r image_utilsr r rrrrprocessing_utilsrrutilsrrrrr;r!r?r<r:rGrOr9rUr^rnr7rr__all__r"r#r$rIs`&  7;O2aa5 |5 $9=#/$/ $S#X 5/*/>'tCH~'%S/' 6 %,,  6*.tCy/ 6 4 tDI ./ 6<8#6 ),*.3"  3" "' 3"sCx4'3"le+eeP r#