^jTddlmZddlmZddlZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZdd lmZdd lmZmZdd lmZdd lmZmZmZmZmZddlmZddlm Z ddl!m"Z"m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z+m,Z,ddl-m.Z.ddl/m0Z0m1Z1ddl2m3Z3ejhe5Z6edeGdde Z7edeGdd eZ8Gd!d"e1Z9Gd#d$e0Z:e d%&Gd'd(e.Z;Gd)d*e jxZ=eGd+d,e+Z>Gd-d.e'Z?Gd/d0e jxZ@d1ejd2ejd3ejd4ejfd5ZBGd6d7e jxZCGd8d9e jxZDeGd:d;eZEdZHd?ejd@eIdAeId4ejfdBZJedCGdDdEeEZKgdFZLy)G)Callable) dataclassN)strict)nn pad_sequence)PreTrainedConfig)create_bidirectional_mask)FlashAttentionKwargs)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutput TensorTypeauto_docstringcan_return_tuplelogging)no_inherit_decorator)requires)CONFIG_MAPPING AutoConfig)AutoModelForKeypointDetection)CLIPMLP)apply_rotary_pos_emb)LlamaAttentioneager_attention_forward)SuperGlueImageProcessorPil)SuperGlueImageProcessorSuperGlueImageProcessorKwargs)SuperPointConfigzETH-CVG/lightglue_superpoint) checkpointceZdZUdZdZdeiZdZee zdze d<dZ e e d<dZ e e d<d Ze e d <dZe dze d <d Zee d <dZee d<dZee d<dZee d<dZee d<dZee ze d<dZee d<fdZdZxZS)LightGlueConfigaG keypoint_detector_config (`Union[AutoConfig, dict]`, *optional*, defaults to `SuperPointConfig`): The config object or dictionary of the keypoint detector. descriptor_dim (`int`, *optional*, defaults to 256): The dimension of the descriptors. depth_confidence (`float`, *optional*, defaults to 0.95): The confidence threshold used to perform early stopping width_confidence (`float`, *optional*, defaults to 0.99): The confidence threshold used to prune points filter_threshold (`float`, *optional*, defaults to 0.1): The confidence threshold used to filter matches Examples: ```python >>> from transformers import LightGlueConfig, LightGlueForKeypointMatching >>> # Initializing a LightGlue style configuration >>> configuration = LightGlueConfig() >>> # Initializing a model from the LightGlue style configuration >>> model = LightGlueForKeypointMatching(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` lightgluekeypoint_detector_configNdescriptor_dim num_hidden_layersnum_attention_headsnum_key_value_headsgffffff?depth_confidencegGz?width_confidence皙?filter_thresholdg{Gz?initializer_rangegelu hidden_actattention_dropoutTattention_biasc |j|j|_t|jtrX|jj dd|jd<t |jddi|jddi|_n |jt dd|_|jdz|_|j|_ t|,di|y)N model_type superpointattn_implementationeager)r<r) r.r- isinstancer'dictgetrr)intermediate_size hidden_sizesuper __post_init__)selfkwargs __class__s z/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/lightglue/modular_lightglue.pyrEzLightGlueConfig.__post_init__Zs  # # +'+'?'?D $ d33T ::>:W:W:[:[\hjv:wD ) ), 7,:4;X;XYe;f,g-//-EL-D ) * * 2,:<,H]d,eD )!%!4!4q!8.. ''cR|j|jzdk7r tdy)zOPart of `@strict`-powered validation. Validates the architecture of the config.rz1descriptor_dim % num_heads is different from zeroN)r)r- ValueError)rFs rIvalidate_architecturez%LightGlueConfig.validate_architecturels,   !9!9 9Q >PQ Q ?rJ)__name__ __module__ __qualname____doc__r:r sub_configsr'r@r"__annotations__r)intr+r-r.r/floatr0r2r3r5strr7r8boolrErM __classcell__rHs@rIr%r%,s6J-z:K?Cd%55<CNCs  &*t*"e""e"!e!#u#J%(us{(ND($RrJr%a Base class for outputs of LightGlue keypoint matching models. Due to the nature of keypoint detection and matching, the number of keypoints is not fixed and can vary from image to image, which makes batching non-trivial. In the batch of images, the maximum number of matches is set as the dimension of the matches and matching scores. The mask tensor is used to indicate which values in the keypoints, matches, matching_scores and prune tensors are keypoint matching information. ) custom_introc^eZdZUdZdZej dzed<dZej dzed<dZ ej dzed<dZ ej dzed<dZ ejdzed<dZ ej dzed<dZeej dzed <dZeej dzed <y) LightGlueKeypointMatchingOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*): Loss computed during training. matches (`torch.FloatTensor` of shape `(batch_size, 2, num_matches)`): Index of keypoint matched in the other image. matching_scores (`torch.FloatTensor` of shape `(batch_size, 2, num_matches)`): Scores of predicted matches. keypoints (`torch.FloatTensor` of shape `(batch_size, num_keypoints, 2)`): Absolute (x, y) coordinates of predicted keypoints in a given image. prune (`torch.IntTensor` of shape `(batch_size, num_keypoints)`): Pruning mask indicating which keypoints are removed and at which layer. mask (`torch.BoolTensor` of shape `(batch_size, num_keypoints)`): Mask indicating which values in matches, matching_scores, keypoints and prune are keypoint matching information. hidden_states (`Tuple[torch.FloatTensor, ...]`, *optional*): Tuple of `torch.FloatTensor` (one for the output of each stage) of shape `(batch_size, 2, num_channels, num_keypoints)` returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True` attentions (`Tuple[torch.FloatTensor, ...]`, *optional*): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, 2, num_heads, num_keypoints, num_keypoints)` returned when `output_attentions=True` is passed or when `config.output_attentions=True` Nlossmatchesmatching_scores keypointsprunemask hidden_states attentions)rNrOrPrQr]torch FloatTensorrSr^r_r`ra IntTensorrbrctuplerdr>rJrIr\r\rs0&*D%  d ")(,GU   %,04OU&&-4*.Iu  4'.$(E5??T !(%)D%  d ")59M5**+d2926Je''(4/6rJr\c eZdZy)LightGlueImageProcessorKwargsN)rNrOrPr>rJrIrjrjsrJrjc deZdZ ddddeeezdedeeee jfffd Z xZ S)LightGlueImageProcessoroutputsr\ target_sizes thresholdreturnc&t||||SNrDpost_process_keypoint_matchingrFrmrnrorHs rIrtz6LightGlueImageProcessor.post_process_keypoint_matchings w5g|YWWrJr6) rNrOrPrlistrhrUr@rVreTensorrtrXrYs@rIrlrlsZ  X2X!4;.X X d3 $% & XXrJrlrebackendsc heZdZed d dddeeezdedeee dfffd Z xZ S) LightGlueImageProcessorPilryrzrmr\rnrorpz torch.Tensorc&t||||Srrrsrus rIrtz9LightGlueImageProcessorPil.post_process_keypoint_matchingsw5g|YWWrJrv) rNrOrPrrrwrhrUr@rVrtrXrYs@rIr}r}sc z"  X2X!4;.X X d3&' ( X#XrJr}c eZdZdeffd Z ddej dedzdeej eej ej fzfdZ xZ S) LightGluePositionalEncoderconfigct|tjd|j|j zdzd|_y)NrFbias)rD__init__rLinearr)r- projectorrFrrHs rIrz#LightGluePositionalEncoder.__init__s: 1f&;&;v?Y?Y&Y]^&^ejkrJr`output_hidden_statesNrpc|j|}|jdd}tj|}tj|}||f}|r||f}|S|f}|S)Nrdim)rrepeat_interleaverecossin)rFr`rprojected_keypoints embeddingscosinessinesoutputs rIforwardz"LightGluePositionalEncoder.forwardsq#nnY7(::1":E ))J' *%u% 6J*12 R\P] rJF) rNrOrPr%rrerxrWrhrrXrYs@rIrrs]ll LQ  =AD[  u|| uU\\5<<%?@ @ rJrc.eZdZdedeffd Z d dejdeejejfdzdejdzdejdzd ejdzd e e d eejejdzffd Z xZ S)LightGlueAttentionr layer_idxc&t||`yrr)rDr rotary_embrFrrrHs rIrzLightGlueAttention.__init__s  OrJNrcposition_embeddingsattention_maskencoder_hidden_statesencoder_attention_maskrGrpc |jdd}g|d|j}|j|j|j dd} |du} | r|n|} | r|n|} |j | j|j dd} |j | j|j dd}||\}}t| | ||\} } tj|jjt}||| | || f|jsdn |j|jd|\}}|j g|dj#}|j%|}||fS)Nrrr6)dropoutscaling)shapehead_dimq_projview transposek_projv_projrr get_interfacer_attn_implementationrtrainingr7rreshape contiguouso_proj)rFrcrrrrrG input_shape hidden_shape query_statesis_cross_attentioncurrent_statescurrent_attention_mask key_states value_statesrrattention_interface attn_output attn_weightss rIrzLightGlueAttention.forwards$))#2. 88b8$--8 {{=166|DNNqRST 2$>2D.-;M!7Sa[[055lCMMaQRS {{>277 EOOPQSTU  **HC';L*VY[^'_ $L*(?(M(M KK , ,.E) %8     " % $}}C$2H2HLL %  % ! \*k));;;;FFH kk+. L((rJ)NNNN) rNrOrPr%rTrrerxrhrr rrXrYs@rIrrs3IM.2596: *)||*)#5<<#=>E*) t+ *) %||d2 *) !& t 3 *)-.*) u||U\\D00 1*)rJrc\eZdZdeffd Zdej dej fdZxZS) LightGlueMLPrct||tj|j|j|_tj |jd|_y)NT)elementwise_affine)rDrrrrBfc1 LayerNorm layer_normrs rIrzLightGlueMLP.__init__sG  99V55v7O7OP,,v'?'?TXYrJrcrpc|j|}|j|}|j|}|j|}|Srr)rr activation_fnfc2)rFrcs rIrzLightGlueMLP.forwardsB/  6 **=9 / rJ rNrOrPr%rrerxrrXrYs@rIrrs,ZZ U\\ellrJrceZdZdedeffd Z d dejdejdejdedzd edzd e eje ejdze ejdzff d Z xZ S) LightGlueTransformerLayerrrct|t|||_t ||_t|||_t ||_yrr)rDrrself_attentionrself_mlpcross_attention cross_mlprs rIrz"LightGlueTransformerLayer.__init__ sD 0C$V, 1&)D%f-rJ descriptorsr`rrNoutput_attentionsrpc|rdnd}|rdnd}|r||fz}|j\}} } |j||||\} } tj|| gd} |j | }||z}|r| |f}|j dd| | j dj || | }|6|j dddd| j dj |dd| nd}|j||||\}}tj||gd}|j|}||z}|r6||f}||j || | fzz|j || | fz|z}|r || fz|fz}|||fS)Nr>)rrrrrrr)rrr) rrrecatrrfliprr)rFrr`rrrall_hidden_statesall_attentions batch_size num_keypointsr)attention_outputself_attentionsintermediate_states output_statesself_attention_descriptorsself_attention_hidden_statesrrcross_attention_outputcross_attentionscross_intermediate_statescross_output_statescross_attention_hidden_statess rIrz!LightGlueTransformerLayer.forwards #7BD0d  1[N B 4?4E4E1 M>-1,?,?  ))/ -@- )/ $ii6F(GRP &9: %0=%@" ,?+O ( ' . .r1m^ T T!W WZ ? )  " "2q!Q > C CA F N Nz[\^_an o 483G3G &"7#9/ 4H4 0 0 %*II/IKa.bhj$k!"nn-FG03FF -FH[,\ )!-55j-Q_`bc./&&z=.QST0 0  +.@@DTCVVN-~==rJ)FF) rNrOrPr%rTrrerxrWrhrrXrYs@rIrr s..3.-2). H>\\H><<H> H> #Tk H>  $; H> u||U5<<047u||9Lt9SS TH>rJr similarity matchability0 matchability1rpc|j\}}}tjj|tjj|j ddz}tjj |d}tjj |j ddj dj dd}|j||dz|dzfd} ||z|z| ddd|d|f<tjj|jd | dddddf<tjj|jd | dddddf<| S)z;create the log assignment matrix from logits and similarityrrrrN) rr functional logsigmoidr log_softmaxrnew_fullsqueeze) rrrrnum_keypoints_0num_keypoints_1 certaintiesscores0scores1scoress rIsigmoid_log_double_softmaxr^sS4>3C3C0J--**=9BMM--k: % T5H5HQ^QeQe(fjn(nn %--jAoq-Q_` &q!t,&q!t,#n&>&>r2&FF  << aMBDAJ((,EAJ((,66r2>E5=D#// 5;;zGWGW;X;\;\]J((5 #++J!OQ qQ %ad+%ad+,JW rJc|j|}tjj|j d}|S)z0Get matchability of descriptors as a probabilityr)rrrsigmoidr)rFrrs rIget_matchabilityz.LightGlueMatchAssignmentLayer.get_matchabilitys7((5 }},,\:BB2F rJ) rNrOrPr%rrerxrr rXrYs@rIrrmsRII5<<u|| 4ELLU\\rJrc\eZdZdeffd Zdej dej fdZxZS)LightGlueTokenConfidenceLayerrclt|tj|jd|_y)Nr)rDrrrr)tokenrs rIrz&LightGlueTokenConfidenceLayer.__init__s& YYv44a8 rJrrpc|j|j}tjj |j d}|S)Nr)rdetachrrr r)rFrrs rIrz%LightGlueTokenConfidenceLayer.forwards= ;--/0 %%e,44R8 rJrrYs@rIrrs*99 5<<ELLrJrc4eZdZUdZeed<dZdZdZdZ dZ dZ y) LightGluePreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. rr& pixel_values)imageFTN) rNrOrPrQr%rSbase_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_supports_flash_attn_supports_sdpar>rJrIrrs1 #$O!&+#NrJrrroc6|j\}}}|ddddddfjd}|ddddddfjd}|j}|j}tj|jd|j d}tj|jd|j d} ||j d|k(} | |j d|k(} |jj}|jd} tj| || } tj| | j d|| }| | |kDz}| |j d|z}tj||d}tj||d}tj||gjddj|dzd}tj| |gjddj|dzd}||fS)z1obtain matches from a score matrix [Bx M+1 x N+1]Nrrrrr)rmaxindicesrearangergathervaluesexp new_tensorwherestackrr)rror_max0max1matches0matches1indices0indices1mutual0mutual1zeromatching_scores0matching_scores1valid0valid1r^r_s rIget_matches_from_scoresr5s||J1 !SbS#2#+  " "1 %D !SbS#2#+  " "1 %D||H||H||HNN1-hooFtLH||HNN1-hooFtLH(//!X66G(//!X66G ;;?? D ??1 D{{7D$7{{7,<,C,CAx,PRVW (94 5F v}}Q1 1F{{68R0H{{68R0Hkk8X./99!Q?GG UVXZ[Gkk#35E"FGQQRSUVW__`jmn`nprsO O ##rJr`heightwidthctj||g|j|jd}|dz }|j dj dz }||ddddfz |dz }|S)a Normalize keypoints locations based on image image_shape Args: keypoints (`torch.Tensor` of shape `(batch_size, num_keypoints, 2)`): Keypoints locations in (x, y) format. height (`int`): Image height. width (`int`): Image width. Returns: Normalized keypoints locations of shape (`torch.Tensor` of shape `(batch_size, num_keypoints, 2)`). rrNrr.).NN)rerrrrr")r`r6r7sizeshiftscales rInormalize_keypointsr=sp << 0@0@  XY] ^D 1HE HHRL  ! #EU3a<00E/4JJI rJzV LightGlue model taking images as inputs and outputting the matching of them. ceZdZdZdeffd ZdedefdZ d#de jde jd e d zde e je e je jfffd Z d e jded e jde jde jf dZd$dZde jde jdede jfdZde jde jd e jde jde jd e jdefdZdZde jde jde jde jde e je jff dZ d%de jde jdeded e jd zde d zd e d zde e je je je e ffdZee d%d e j.d!e j0d zde d zd e d zde ezf d"ZxZS)&LightGlueForKeypointMatchingar LightGlue is a model matching keypoints in images by leveraging detections from a keypoint detector such as SuperPoint. It is based on the SuperGlue architecture and is designed to be lightweight and efficient. It consists of : 1. Keypoint Encoder 2. A Graph Neural Network with self and cross attention layers 3. Matching Assignment layers The correspondence ids use -1 to indicate non-matching points. Philipp Lindenberger, Paul-Edouard Sarlin and Marc Pollefeys. LightGlue: Local Feature Matching at Light Speed. In ICCV 2023. https://huggingface.co/papers/2306.13643 rc t||tj|j|_|jj |_|j|_|j|_ |j|_ |j|_ |j|_ |j|jk7r2tj|j|jd|_ntj"|_t%||_tj(t+|jDcgc]}t-||c}|_tj(t+|jDcgc] }t1|c}|_tj(t+|jdz Dcgc] }t5|c}|_|j9ycc}wcc}wcc}w)NTr)rr)rDrr from_configr'keypoint_detectordescriptor_decoder_dim keypoint_detector_descriptor_dimr)r+ num_layersr2r/r0rrinput_projectionIdentityrpositional_encoder ModuleListrangertransformer_layersrmatch_assignment_layersrtoken_confidence post_init)rFrir'rHs rIrz%LightGlueForKeypointMatching.__init__s  !>!J!J6KjKj!k060O0O0f0f-$33 22 & 7 7 & 7 7 & 7 7   $"G"G G$&IId.S.SUYUhUhos$tD !$&KKMD !">>rJkeypoint_confidencesrb num_pointsc||j\}}||jdz kru|j|dk(d}|j|dzd}|j |}d||kj j d|z z }||jkD} | Stj|tj} | S)zRevaluate whether we should stop inference based on the confidence of the keypointsrrrrg?rr) rrErrrTrUsumr/reonesrW) rFrZrPrbr[rr'roratio_confidentearly_stopped_pairss rI_get_early_stopped_image_pairsz;LightGlueForKeypointMatching._get_early_stopped_image_pairs's A 1, ,$8#C#CDAIq#Q #7#?#? aQS#T 66{CI!%9I%E$L$L$N$R$RWX$R$Y\f$ffO"1D4I4I"I  #"#(**Zuzz"J ""rJc| ||}||}|j|||}t||j\}}||fSrr)rLr5r2)rFrrbrP early_stopsrr^r_s rI_get_keypoint_matchingz3LightGlueForKeypointMatching._get_keypoint_matching:sW  "%k2K $D:--k:;M#:64CXCX#Y ''rJ confidencesrc\|d|jz kD}||||j|kz}|S)z#mask points which should be removedr)r0rT)rFrfrrPkeeps rI_get_pruning_maskz.LightGlueForKeypointMatching._get_pruning_maskBs<T2223  " K4#A#A+#NN ND rJr prune_outputc|j\}} } |j|j|} |j|| |j |dk(t j dfd||d|d|fD\} } } }}t|D]}||||fxxdz cc<d| | | |fD\} } } }| | f}t|dd}| ||||fS) z For a given layer, prune keypoints based on the confidence of the keypoints and the matchability of the descriptors. rFc3nK|]&}t|Dcgc] \}}|| c}}(ycc}}wwrr)zip).0rtrbpruned_keypoints_masks rI zJLightGlueForKeypointMatching._do_layer_keypoint_pruning..]s9c %(0E$F GDQtW Gc Gs 5/5rc36K|]}t|dyw)T) batch_firstNr)rn pruned_tensors rIrqzJLightGlueForKeypointMatching._do_layer_keypoint_pruning..es$S  D 9 9S sTrrs padding_value) rrLr rirrerrJr)rFrr`rbrrjrZrPrr'descriptors_matchabilitypruned_descriptorspruned_keypoints_0pruned_keypoints_1 pruned_maskpruned_indicesrOpruned_keypointsrps @rI_do_layer_keypoint_pruningz7LightGlueForKeypointMatching._do_layer_keypoint_pruningIs+',, Aq#'#?#? #L#]#]^i#j $ 6 67KMegr s 5 A A$!)U\\Z_M` ac & ! ilDY[bcc _.0BKQ_z" 4A N1-- .! 3 . 4S "46HJ\^i!jS O.0BK/0BC%n$VXY!#3^[R^^^rJctjtjjd}|}|d||fD\}}d||fD\}}fd||||fD\}}}}||||fS)Nrc38K|]}t|ddyw)TrruNrrnrs rIrqzMLightGlueForKeypointMatching._concat_early_stopped_outputs..{s$3  T D D3 c38K|]}t|ddyw)TrruNrrs rIrqzMLightGlueForKeypointMatching._concat_early_stopped_outputs..s$>  T C C> rc3(K|] }| ywrrr>)rnrearly_stops_indicess rIrqzMLightGlueForKeypointMatching._concat_early_stopped_outputs..s!g  & 'g s)rer&r r)rFrfinal_pruned_keypoints_indices!final_pruned_keypoints_iterationsr^r_ids order_indicess ` rI_concat_early_stopped_outputsz:LightGlueForKeypointMatching._concat_early_stopped_outputsns$kk*=>ll.44Q78+C0 1-@3 "$BC3 //> *,MN> ::g .1 g c"@Bc./PRY[jjjrJr^r_rc |j\}fd|||fD\}}}|dddf}|dddf}|dddf}|dddf} |dddf} |dddf} tjdzd|fd|j|j} tj dzd|f|j|j} t dzD]}tj||dk(d||jd||jd| |d||f<tj| |dk(d||jd| |jd| |d||f<| || |d||f<| || |d||f<| | fS)Nc3JK|]}|jdzddyw)rrN)r)rnrrs rIrqzJLightGlueForKeypointMatching._do_final_keypoint_pruning..s'- 7=FNN:?Ar 2- s #rrrrr9)r) rrefullrrzerosrJr%r!clamp)rFrr^r_rr'r,r-r*r+r1r2_matches_matching_scoresrOrs @rI_do_final_keypoint_pruningz7LightGlueForKeypointMatching._do_final_keypoint_prunings A- BI7TcAd- )/1a4=1a4=1a4=1a4=*1a40*1a40::zQ=A2gnndkdqdqr ;; 1_a /oNcNc zQ' FA*/++ r!2x{'9'9!Xa[=N=NST=N=U'V+HQ8A;& '+0++ r!2x{'9'9!Xa[=N=NST=N=U'V+HQ8A;& '3C12E Q8A;. /2B12E Q8A;. / F)))rJr6r7rc J '|rdnd}|rdnd} |jddk(rT|jdd} |j| dtj|j | |j | || fS|j } |j\} } }} tj |j| dd}|j| dz|d}||j| dz|nd}|j| dz||j}tj| dz| }t|||}|j||| \}}|d}|jdkD}|jdkD}g}g}g}g}g}tjd|| j| dzd}tj|}t!|j"D]E}|'t%|j&|ddddddf|d }n4tj(| |j+d f|j }|j,|||||| }|\}}}|r||z}|r| |z} |r||j"dz kr+|j.||} |j1| |||}!n%tj(| tj2}!tj4|!r|!j7d'|'}"|j9|||'\}#}$|j;t=|"|j;t=|#|j;t=|$|r:|j;t=|'|j;t=|'||!}t?'fd||d|d||fD\}}%}&}}|%|&f}|rt?'fd|| fD\}}} tj@|!rn$|s)|jC||||| |\}}}}}H|r4|r2|jE|||||\}}}}|jG||||\}}nE|j9|||j"dz \}}tj||j"z}|j| d|}||||| fS)Nr>rrrr]rrrrVcLtjdtjS)NTr])rerrW)argss rIz@LightGlueForKeypointMatching._match_image_pair..sELLUZZ4XrJ)r inputs_embedsrand_mask_functionr)rrr)r[)rdc3*K|] }| ywrrr>rnrrds rIrqzALightGlueForKeypointMatching._match_image_pair..#s"V" |,Vc3*K|] }| ywrrr>rs rIrqzALightGlueForKeypointMatching._match_image_pair..)s"l &#K<0lr)$rrrerT new_zerosrr^rrDr r=rYr/r0expand ones_likerJrEr rr_r:rKrMrbrWanyrreextendrwrhallr~rr)(rFr`rr6r7rbrrrrrrrr'initial_num_keypointsnum_points_per_pair image_indicesrX do_early_stopdo_keypoint_pruningrr^r_rrpruned_keypoints_indicespruned_keypoints_iterationsrPextended_attention_mask layer_outputrc attentionrZraearly_stopped_image_indicesearly_stopped_matchesearly_stopped_matching_scores keypoints_0 keypoint_1rds( @rI_match_image_pairz.LightGlueForKeypointMatching._match_image_pairs|#7BD0d ??1  "OOCR(E""5"EII">##E*##E*!  !!2;/// A,a#ii Z(D!L%%j1n6KQO FJFVt||JN,AB\`!))*q.:OQUQvQvw  Z!^FC ' 65A 040I0I 9M1J1 - --Q/ --1 #33a7 )+&,.)#(<<3HQW#X#_#_`jmn`npr#s &+oo6N&O# 1X K*C;;"-a1ai"8#'&X +'+0**j+BRBRBTUWBX5Ybkbrbr*s'?422;?6%9"3 L5A 1K #$5 $E! !/)!;1!44+M4+@+@+Mk+Z(+/*M*M,k4L_+N+' +0**Zuzz*R'9901#6"G"G"JK2? 2L/KOKfKf#T;KLgLH)+H(..t4O/PQNN4(=#>?#**40M+NO*6==dC[\gCh>ij9@@FabmFnAop+>?R>R*S'PUV'2IaL)A,PTVc&dVQMKj$ "-j 9I*fkl!9 ; 4+lgc02MOc9901"33#!03,#d Y(@$Hc]X t 022'25#  h *,MwXg(,'F'F.% ( $G_(,'B'B;PTVZVeVehiVi'j $G_050PSWSbSb0b -,M,U,U 0- )   -     rJrlabelsc 0d}| td||n|jj}||n|jj}|jdk7s|j ddk7r td|j \}}} } } |j|dz| | | }|j|} | dd\} }}}| j|dddj|} |j|dd|jj|}|j|dd}| j}|dddddddf| z|dddddddf<|dddddddf| z|dddddddf<|j||| | ||| \}}}}}t|||| |||| S) Nz9LightGlue is not trainable, no labels should be provided.rrzOInput must be a 5D tensor of shape (batch_size, 2, num_channels, height, width)r,rr)rbrr)r]r^r_r`rarbrcrd)rLrrrndimr:rrrBtorDclonerr\)rFrrrrrGr]rr'channelsr6r7keypoint_detectionsr`rrbabsolute_keypointsr^r_rarcrds rIrz$LightGlueForKeypointMatching.forwardfs  XY Y1B1N-TXT_T_TqTq$8$D $++JjJj     !\%6%6q%9Q%>no o1=1C1C. Ax#++JNHfeT "44\B*=bq*A' 1k4%%j!R;>>|L !))*aT=b=bcffgst ||J2.&__.);Aq!QJ)G%)O1aA:&);Aq!QJ)G&)P1aA:&EIE[E[    /!5F\F B% /+'!  rJrrr)NNN)rNrOrPrQr%rrTrUrTrerxrWrhrYrbrerir~rrrrrrf LongTensorr\rrXrYs@rIr?r?s <(S(U( gl? <<?49LL?X\_cXc? u||U5<<#=>> ??#$)LL#?B#JO,,#didpdp# #&(U\\5<<^afkfrfr#_\\#_<<#_ll #_  #_ ll #_$ll#_#_Jk@#*#*#* #* || #* u||U\\) * #*V%))-,0q <<q \\q  q  q llT! q  $;q #Tkq  u||U\\5<<E Fq f+/)-,0 4 ''4   4'4  $; 4 #Tk 4  0 04 4 rJr?)rr?r%rlr})Mcollections.abcr dataclassesrnumpyrRrehuggingface_hub.dataclassesrrtorch.nn.utils.rnnrconfiguration_utilsr masking_utilsr modeling_flash_attention_utilsr modeling_utilsr rprocessing_utilsrutilsrrrrr utils.genericrutils.import_utilsrautorrauto.modeling_autorclip.modeling_cliprcohere.modeling_cohererllama.modeling_llamarr(superglue.image_processing_pil_supergluer$superglue.image_processing_supergluer r!r;r" get_loggerrNloggerr%r\rjrlr}ModulerrrrrxrrrrrUrhr5rTr=r?__all__r>rJrIrsN%! .+36BF&WW1*->(9JQi)   H %9:AR&AR;ARH  7k 7  7F $A X5X :X!;XX"/)/)/)d 7 P> P>f   -2\\ JO,,  \\ &BII&R BII    $ELL$U$uU\\[`[g[gMgGh$@5<<SU\\, o #;o  o d  rJ