^j14ddlmZddlmZmZddlmZddlm Z ddl m Z ddl m Z mZddlmZmZmZmZmZmZmZmZdd lmZmZdd lmZmZmZerdd lZerd d l m!Z!GddedZ"dZ#defdZ$ ddZ% ddZ&eGdde Z'dgZ(y )) TYPE_CHECKING)Image ImageDraw) functional)TorchvisionBackend) BatchFeature)group_images_by_shapereorder_images) ImageInput ImageTypePILImageResamplingSizeDictget_image_type is_pil_imageis_valid_imageto_numpy_array) ImagesKwargsUnpack) TensorTypeauto_docstringis_torch_availableN)$EfficientLoFTRKeypointMatchingOutputceZdZUdZeed<y)"EfficientLoFTRImageProcessorKwargsz do_grayscale (`bool`, *optional*, defaults to `self.do_grayscale`): Whether to convert the image to grayscale. Can be overridden by `do_grayscale` in the `preprocess` method. do_grayscaleN)__name__ __module__ __qualname____doc__bool__annotations__/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/efficientloftr/image_processing_efficientloftr.pyrr$s r%rF)totalct|xsCt|xr6t|tjk7xrt |j dk(S)Nrrrrr PILlenshapeimages r&_is_valid_imager/-sF   ub."79=="HbSQVQ\Q\M]abMbr%imagescd}dt|trQt|dk(rtfd|Dr|Stfd|Dr|Dcgc] }|D]}| c}}St |cc}}w)N)z-Input images must be a one of the following :z - A pair of PIL images.z - A pair of 3D arrays.z! - A list of pairs of PIL images.z - A list of pairs of 3D arrays.ct|xsCt|xr6t|tjk7xrt |j dk(S)z$images is a PIL Image or a 3D array.rr)r-s r&r/z8validate_and_format_image_pairs.._is_valid_image<sGE" 5 ! fnU&;y}}&L fQTUZU`U`QaefQf r%c3.K|] }|ywNr$.0r.r/s r& z2validate_and_format_image_pairs..Cs#Q_U%;#Qc3K|]:}t|txr$t|dk(xrtfd|D<yw)r3c3.K|] }|ywr5r$r6s r&r8z..HsCuOE*Cr9N) isinstancelistr+all)r7 image_pairr/s r&r8z2validate_and_format_image_pairs..EsN  z4 ( DJ1$ DC CC D sAA)r<r=r+r> ValueError)r0 error_messager?r.r/s @r&validate_and_format_image_pairsrB3sM &$ v;! #Q&#Q QM  %   -3Kj KuEKEK K ] ##LsA3c $|jdks#|j|jdk(rdnddk(rytj|ddddddf|ddddddfk(xr.tj|ddddddf|ddddddfk(S)zAChecks if an image is grayscale (all RGB channels are identical).rrrT.Nr3)ndimr,torchr>r-s r& is_grayscalerFOs zzA~%**/QqAQF 99U31a<(E#q!Q,,?? @ UYY c1aluS!Q\22Fr%cJt|r|Stj|dS)a Converts an image to grayscale format using the NTSC formula. Only support torch.Tensor. This function is supposed to return a 1-channel image, but it returns a 3-channel image with the same value in each channel, because of an issue that is discussed in : https://github.com/huggingface/transformers/pull/25786#issuecomment-1730176446 Args: image (torch.Tensor): The image to convert. r)num_output_channels)rFtvFrgb_to_grayscaler-s r&convert_to_grayscalerKZs$E   1 ==r%cxeZdZeZej ZdddZdZ dZ dZ dZ dZ dZdeeffd Zed edeed effd Zd ed efd Z d"d eddededddedededzdeezdzded efdZ d#dddeeezded eeeej@ffdZ!deeeej@fd edfd Z"d!Z#xZ$S)$EfficientLoFTRImageProcessorii)heightwidthFTgp?Nkwargsc $t|di|y)Nr$)super__init__)selfrP __class__s r&rSz%EfficientLoFTRImageProcessor.__init__ys "6"r%r0returnc $t||fi|Sr5)rR preprocess)rTr0rPrUs r&rXz'EfficientLoFTRImageProcessor.preprocess|sw!&3F33r%c :|j|}t|Sr5) fetch_imagesrB)rTr0rPs r&_prepare_images_structurez6EfficientLoFTRImageProcessor._prepare_images_structures ""6*.v66r% torch.Tensor do_resizesizeresamplez7PILImageResampling | tvF.InterpolationMode | int | None do_rescalerescale_factordisable_groupingreturn_tensorsrc  (t||\} } i} | jD] \}}|r|j|||}|| |<"t| | }t||\} } i} | jD]+\}}|r|j ||}| r t |}|| |<-t| | }t dt|dDcgc] }|||dz }}|Dcgc]}tj|d}}td|i|Scc}wcc}w)N)rb)r^r_rr3)dim pixel_values)data tensor_type) r itemsresizer rescalerKranger+rEstackr )rTr0r]r^r_r`rarbrcrrPgrouped_imagesgrouped_images_indexprocessed_images_groupedr,stacked_imagesresized_imagesprocessed_imagesi image_pairspair stacked_pairss r& _preprocessz(EfficientLoFTRImageProcessor._preprocesssS0EV^n/o,,#% %3%9%9%; = !E>!%^$QY!Z.< $U + =((@BVW/D^fv/w,,#% %3%9%9%; = !E>!%nn!M!5n!E.< $U +  =**BDXY=B!SIYEZ\]<^_q'AE2_ _?JJdTq1J J.-!@n]]`Ks D Doutputsr target_sizes thresholdc4|jjdt|k7r tdt d|Ds tdt |t r,tj||jj}n1|jddk7s|jddk7r td|}|jj}||jdjddddz}|jtj}g}t!||j|j"D]X\}}} tj$| |kD|dkD} |d| d} |d| d} | d| d} |j'| | | d Z|S) a Converts the raw output of [`EfficientLoFTRKeypointMatchingOutput`] into lists of keypoints, scores and descriptors with coordinates absolute to the original image sizes. Args: outputs ([`EfficientLoFTRKeypointMatchingOutput`]): Raw outputs of the model. target_sizes (`torch.Tensor` or `List[Tuple[Tuple[int, int]]]`, *optional*): Tensor of shape `(batch_size, 2, 2)` or list of tuples of tuples (`Tuple[int, int]`) containing the target size `(height, width)` of each image in the batch. This must be the original image size (before any processing). threshold (`float`, *optional*, defaults to 0.0): Threshold to filter out the matches with low scores. Returns: `List[Dict]`: A list of dictionaries, each dictionary containing the keypoints in the first and second image of the pair, the matching scores and the matching indices. rzRMake sure that you pass in as many target sizes as the batch dimension of the maskc38K|]}t|dk(yw)r3N)r+)r7 target_sizes r&r8zNEfficientLoFTRImageProcessor.post_process_keypoint_matching..sI[3{#q(IszTEach element of target_sizes must contain the size (h, w) of each image of the batch)devicerr3) keypoints0 keypoints1matching_scores)matchesr,r+r@r>r<r=rEtensorr keypointscloneflipreshapetoint32zipr logical_andappend)rTryrzr{image_pair_sizesrresultskeypoints_pairrscores valid_matchesmatched_keypoints0matched_keypoints1rs r&post_process_keypoint_matchingz;EfficientLoFTRImageProcessor.post_process_keypoint_matchings, ??  #s<'8 8qr rILIIst t lD )$||LAWAWX !!!$)\-?-?-Ba-G j , %%++-  0 5 5b 9 A A"aA NN LL- /29goowOfOf/g  +NGV!--fy.@'B,OM!/!2=3C!D !/!2=3C!D $Qi a(89O NN"4"4'6   r%keypoint_matching_outputz Image.Imagec <t|}|Dcgc] }t|}}tdt|dDcgc] }|||dz }}g}t ||D]\}}|dj dd\} } |dj dd\} } t jt| | | | zdft j} t j|d| d| d| f<t j|d| d| | df<tj| j}tj|}|dj!d\}}|dj!d\}}t |||||d D]\}}}}}|j#|}|j%|||| z|f|d |j'|dz |dz |dz|dzfd |j'|| zdz |dz || zdz|dzfd |j)||Scc}wcc}w) a Plots the image pairs side by side with the detected keypoints as well as the matching between them. Args: images: Image pairs to plot. Same as `EfficientLoFTRImageProcessor.preprocess`. Expects either a list of 2 images or a list of list of 2 images list with pixel values ranging from 0 to 255. keypoint_matching_output (List[Dict[str, torch.Tensor]]]): A post processed keypoint matching output Returns: `List[PIL.Image.Image]`: A list of PIL images, each containing the image pairs side by side with the detected keypoints as well as the matching between them. rr3Nrr)dtyperrr)fillrOblack)r)rBrrlr+rr,rEzerosmaxuint8 from_numpyr fromarraynumpyrDrawunbind _get_colorlineellipser)rTr0rr.rtrurr? pair_outputheight0width0height1width1 plot_imageplot_image_pildraw keypoints0_x keypoints0_y keypoints1_x keypoints1_y keypoint0_x keypoint0_y keypoint1_x keypoint1_ymatching_scorecolors r&visualize_keypoint_matchingz8EfficientLoFTRImageProcessor.visualize_keypoint_matchingsh(185;>.1D)4\)B)I)I!)L &L,)4\)B)I)I!)L &L,VYlL, TeHfW R [+{N7  +{V/C[Q  kAo{Q aQ\_`Q`aho p  6)A-{Q f@TWX@XZehiZij   NN> *7 +8A=Ks HHcNtdd|z z}td|z}d}|||fS)zMaps a score to a color.rr)int)rTscorergbs r&rz'EfficientLoFTRImageProcessor._get_color&s3 q5y! " e  !Qwr%)T)g)%rrr r valid_kwargsrBILINEARr_r^default_to_squarer]r`ra do_normalizerrrSrr r rXr[r=r"rfloatstrrrxtupledictrETensorrrr __classcell__)rUs@r&rMrMms5L!**HC (DIJNLL#(J!K#44v>`7a4fr4477  7&")^^$)^)^ )^ L )^  )^)^+)^j(4/)^)^ )^^ 979!4;.9 9 d3 $% & 9v5#'tC,='>"?5 m  5nr%rM)r.r\)r.r\rVr\))typingrr*rrtorchvision.transforms.v2rrIimage_processing_backendsrimage_processing_utilsr image_transformsr r image_utilsr r rrrrrrprocessing_utilsrrutilsrrrrEmodeling_efficientloftrrrr/rBrFrKrM__all__r$r%r&rs! 7;2E   5CCMU $J$8 > >>&}#5}}@ * *r%