^jeWUdZddlmZddlmZddlmZddlZddlm cm Z ddlm Z ddl m Z e ZdZd ed <dd ZGd d e j$Z ddZddZ ddZgdZy)uKeypoint helper primitives for GroupPose-style decoding. Keypoint predictions are dense ``[N, K, KEYPOINT_PRED_DIM]`` tensors where the trailing channel layout is fixed: ================ =========== ========================================================== Slot index Name Meaning ================ =========== ========================================================== ``0`` ``x`` Normalized x coordinate (image-relative). ``1`` ``y`` Normalized y coordinate (image-relative). ``2`` ``findable`` Logit for "annotator could find this keypoint" (``v > 0``). ``3`` ``visible`` Logit for "fully visible" (``v == 2``). ``4`` – ``6`` ``L_*`` Lower-triangular Cholesky parameters ``Lxx``, ``Lxy``, ``Lyy``. ``7`` ``class`` Per-keypoint class-logit contribution aggregated into detection-class logits. ================ =========== ========================================================== ) annotations)Sequence)castN)nn) get_loggerintKEYPOINT_PRED_DIMc|dz|z|zS)u$Apply AdaLN modulation to a feature tensor. Args: features: Input feature map to modulate. scale: Per-feature scale terms. shift: Per-feature shift terms. Returns: Modulated features with the same shape as ``features``. Example: Apply modulation to a batch of 2 query tokens, each with 4 channels: .. code-block:: python features = torch.zeros(2, 4) # (batch, dim) scale = torch.ones(2, 4) * 0.1 # small positive scale shift = torch.ones(2, 4) * (-0.5) # constant shift out = modulate(features, scale, shift) # scale=0.1 → effective multiplier is 1.1; shift=-0.5 applied additively # out[0] ≈ tensor([-0.5000, -0.5000, -0.5000, -0.5000]) g?)featuresscaleshifts h/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/rfdetr/models/heads/keypoints.pymodulater's2 CK8 #e ++c.eZdZdZddfd ZddZxZS)ConditionalQueryInitializerzTInitialize keypoint query tokens with adaptive layer-normalization style modulation.c t||xs|}tjt j |||_tj|d|_tjtj||tjtj||dz|_ ttj|jd}tjj|j dtjj|j"dtj|||_y)zCreate the initializer. Args: dim: Query conditioning dimensionality. num_queries: Number of query tokens to instantiate. out_dim: Output embedding size. Defaults to ``dim``. F)elementwise_affinerN)super__init__r Parametertorchrandnqueries LayerNorm query_norm SequentialLinearGELUadaLN_modulationrinit constant_weightbiasout_proj)selfdim num_queriesout_dimada_ln_projection __class__s rrz$ConditionalQueryInitializer.__init__Fs .S||EKK W$EF ,,w5I " IIc3  GGI IIc7Q; '!  !D,A,A",EF +22A6 +00!4 '73 rc2|j|j}|j|jd}|j dd\}}}|j t ||||z|jz}ttj|S)axReturn modulated query embeddings. Args: query_features: Tensor of shape ``(B, dim)`` containing conditioning features. Returns: Tensor of shape ``(B, num_queries, out_dim)`` with initialized keypoint queries. Example: Initialize 17 keypoint queries conditioned on a batch of 2 detection embeddings: .. code-block:: python initializer = ConditionalQueryInitializer(dim=256, num_queries=17) query_features = torch.randn(2, 256) # (B=2, dim=256) keypoint_queries = initializer(query_features) # keypoint_queries.shape == (2, 17, 256) # Each of the 17 query slots is independently modulated by the # per-detection conditioning vector before being passed to the # GroupPose keypoint decoder. rrr+) r rr$ unsqueezechunkr)rrrTensor)r*query_featuresnormed_query_features modulationrrgatemodulated_query_featuress rforwardz#ConditionalQueryInitializer.forward^s0!% =#'#8#89Q9QRT9U#V '--aR-8ud#'==:OQVX]1^#_bf#fimiuiu#u ELL":;;r)N)r+r r,r r-z int | NonereturnNone)r6 torch.Tensorr<r>)__name__ __module__ __qualname____doc__rr; __classcell__)r/s@rrrCs^40=7)``. target_keypoints: Ground truth keypoints with shape ``(N, K_max, 3)``. target_classes: Class ids per target with shape ``(N,)``. target_areas: Target box areas with shape ``(N,)``. num_keypoints_per_class: Number of keypoints per class. Returns: Tuple of location, findable BCE, visible BCE, and raw Gaussian NLL losses. Each tensor has shape ``(n_targets,)``. Example: Compute losses for 2 matched targets, each with 17 keypoints (COCO layout): .. code-block:: python n_targets, K, pred_dim = 2, 17, 7 all_pred_keypoints = torch.randn(n_targets, K, pred_dim) # Ground truth: 17 keypoints per target, each (x, y, visibility) # visibility: 0=not labeled, 1=labeled but occluded, 2=fully visible target_keypoints = torch.rand(n_targets, K, 3) target_keypoints[:, :, 2] = 2.0 # mark all keypoints fully visible target_classes = torch.zeros(n_targets, dtype=torch.long) # single class (person) target_areas = torch.tensor([0.05, 0.12]) # normalized box areas loc_loss, findable_loss, visible_loss, nll_loss = compute_l1_keypoint_loss( all_pred_keypoints, target_keypoints, target_classes, target_areas, num_keypoints_per_class=[17], ) # Each output tensor has shape (2,) — one scalar loss per matched target. # loc_loss: area-normalized mean L1 distance for visible keypoints # findable_loss: BCE for "annotator could locate this keypoint" # visible_loss: BCE for "keypoint is fully visible (v==2)" # nll_loss: Gaussian NLL incorporating Cholesky uncertainty parameters IExpected all_pred_keypoints last dim >= 7 (x,y,2 logits + 3 chol params).rzInum_keypoints_per_class must be non-empty when computing keypoint losses.ztarget_classes max index %d >= num_keypoints_per_class length %d; skipping keypoint loss for this batch to avoid crashing training. Check that your keypoint schema covers all annotation classes.)device)dtyperGTNrrHr2minnone) reductionrzGNLL loss: %d keypoint(s) with non-finite uncertainty dropped from loss.?nanposinfneginf)&shapelen ValueErrormaxloggerwarningr new_zerosviewrarangerGzerosbool enumeratesumtorHfloat32finfoepsisfiniteallr3clamp clamp_minsqrtFl1_loss binary_cross_entropy_with_logitsdebugitemexp nan_to_numrM masked_fillwhere zeros_like)6all_pred_keypointstarget_keypointstarget_classes target_areasnum_keypoints_per_class n_targetstotal_padded_num_keypointspred_dim num_classesrbkpadsplit_pred_keypointsselected_pred_keypointsactive_keypoints_mask class_idx num_keypointskeypoints_loss_maskkeypoints_per_targetareaarea_eps valid_areavalid_xyvalid_visibilitylocation_loss_masklocation_count valid_countdenom_keypointssafe_area_sqrtscaled_masked_l1 location_loss findable_loss visible_lossdxdydxdy raw_log_l11raw_l21 raw_log_l22finite_uncertaintygaussian_loss_masklog_l11l21log_l22l11l22u0u1maha2gaussian_countgaussian_valid_countnll_raw nll_keypointsnll_lossno_valids6 rcompute_l1_keypoint_lossr}sr7I6N6N3I)8 q=eee=-.Kadee1}++-< M ""$ %   #,,Y7eUE)) % 4D-229k4QYZ2 Y';'B'BC^S"KK TThToTop$-.E$F@ =;?i-78@0?.222699@W@]@]9^ ??5== )D{{4::&**H%9J~~5aBQBh?@DDDLu~~Arr"P cbckH~~&6q!Qw&?@DTUVXY[\U\D]`aDab,/??(JZMaMabcMdd'++B/229P9V9V2WN &&1&-K*00Q07::AXA^A^:_O^^H-224N )!Q(35EaBQBh5O[abffgij    7 = = > ?  " "1 % & %((,{:M ** #Aq!G , aAg & * . ./F/L/L M  !8!>!> ?  @  c"g M ** #Aq!G , aAg & * . ./F/L/L M  !8!>!> ?  @  c"g L $Aq"1"H -0@Arr0J J N Nu}} ]D aAgB aAgB)!Q'255emmDK%aAg.11%--@G)!Q'255emmDK 4u~~g7NNQVQ_Q_`kQll  ! ! # U % % ' , , . ,.@@G CG ++-C ++-C rC"H B rB Gb2g E+ennR.@@5>>RTCUUX]XfXfglXmm'++B/229P9V9V2WN)//A/6UT^^H5??BBCwQXGXYGwCEKKPWP]P]D^DbDbcG'');(;SAM  $';;H"H{{8U%5%5h%?JH -x ??rc|j|j}tj|j dd}t j ||j}||z S)z;Compute pairwise BCE-with-logits summed along the last dim.rIrKT)r+keepdim)rfrHrosoftplusrermatmult)xyy_floatrdots r_cdist_bce_with_logitsr#sSddd!Gzz!}  Q 5H ,,q'))+ &C c>rc H|j\}}}}|dk\sJd|jd} t|} | dk(r td| dk(r5tj||| f|j |j } | | | | fS|| z} |j||| | |} tj||| f|j |j }tj||| f|j |j }tj||| f|j |j }tj||| f|j |j }||z}t| D]2}||k(jjd}|jdk(r:||}|dk(rE| dddd|d|ddf}|jd|ddd|ddf}|jd}|jd|jtj}tj|j j }tj"|||kDz}|ddddddf}tj"|dddddf|dddddfdkDztj"|j%d z|j'dz}|j)d jtj}|j+d }|dkD} |j-|j/}!|ddddddddfj1||djtj}"|jtj}#|"j'd|#j'dz }$|$j3j)d }%|j'd}&|%j5|&d }%|%j)d }'|'|j'dz j7|!j'd}(tj8|(d d d }(|(j1|||j|j |dddd|f<|dddddddfj1||jtj})|dddddddfj1||jtj}*|dddddddfj1||jtj}+|)},|*}-|+}.|,j;}/|.j;}0tj"|"j%d }1|1tj"|)ztj"|*ztj"|+z}2|$d}3|$d}4|/j'd|3z|-j'd|4zz}5|0j'd|4z}6|5|5z|6|6zz}7|&|2j'dztj"|5ztj"|6ztj"|7z}8d|7|j-|jd|dz z|,|.zj'dz }9tj8|9d d tj|9j j< }9|9j5|8d }9|9j)d }:|:|j'dz j1|||};|;j?| j'dd |;j|j |dddd|f<|dddddddfj1||}<|dddddfdkDj|j j1||}=|dddddddfj1||}>|dddddfdkDj|j j1||}?tA|<|=j1|||tC|z |dddd|f<tA|>|?j1|||tC|z |dddd|f<5||||fS)uCompute many-to-many keypoint matching costs. Args: all_pred_keypoints: Predicted keypoints of shape ``(B, Q, K_total, >=7)``. target_keypoints: Ground truth keypoints with shape ``(N, Kmax, 3)``. target_classes: Class ids for each target with shape ``(N,)``. target_areas: Target box areas with shape ``(N,)``. num_keypoints_per_class: Number of keypoints per class. Returns: Tuple ``(cost_l1, cost_findable, cost_visible, cost_nll)`` each of shape ``(B, Q, N)``. Example: Compute matching costs for 1 image with 4 decoder queries against 2 ground-truth targets, each with 17 COCO keypoints: .. code-block:: python B, Q, K_total, pred_dim = 1, 4, 17, 7 n_targets = 2 all_pred_keypoints = torch.randn(B, Q, K_total, pred_dim) target_keypoints = torch.rand(n_targets, K_total, 3) target_keypoints[:, :, 2] = 2.0 # mark all keypoints fully visible target_classes = torch.zeros(n_targets, dtype=torch.long) # single class (person) target_areas = torch.tensor([0.05, 0.12]) cost_l1, cost_findable, cost_visible, cost_nll = compute_keypoint_matching_cost( all_pred_keypoints, target_keypoints, target_classes, target_areas, num_keypoints_per_class=[17], ) # Each output tensor has shape (1, 4, 2) — (B, Q, N). # cost_l1[:, q, n] is the area-normalized L1 cost for query q against target n. # cost_nll[:, q, n] incorporates the Cholesky precision uncertainty. # These cost matrices are passed to the Hungarian matcher. rErFrzQnum_keypoints_per_class must be non-empty when computing keypoint matching costs.)rGrHrKNrJrr2rLrTrUrPrQrR).r).rKrSr)"rYrZr[rrbrGrHr`rangenonzerosqueezenumel index_selectrfrgrhrirjrkr3rerlrmrnreshapeabsrvdivrurtrM masked_fill_rfloat)@ryrzr{r|r}br,total_num_keypointsrr~rrbrpredcost_l1 cost_findable cost_visiblecost_nllflat_bqrtarget_indicesnum_kpts pred_by_classtarget_by_classn_targets_by_classareasrr target_xyvisiblevalid_per_target nll_denom has_visible area_sqrt pred_xy_flat target_xy_f32diff per_kpt_l1 visible_btk scaled_locloc_costrrrrrrrr finite_xy finite_predrrrrr keypoint_masknll_knll_summean_nll pred_findabletarget_findable pred_visibletarget_visibles@ rcompute_keypoint_matching_costr+sOb5G4L4L1A{' q=eee= &&q)I-.KalmmA~  Y '%,,$**  eUE)) + -D  " "1k;h ODkk K#!(( &&G KK K#!(( &&M ;; K#!(( &&L {{ K#!(( &&H +oG;'q (I5>>@HHK    !Q & *95 q= Q9ixi:; *77>J1ixiYZ?[,2215))!^<?? N;;u{{+//^^E*eh.>? #Aq"1"H- NN?1a73 4q!Qw'!+ -nnY'+++3 4""1% &  #;;1;-00?$**q*1 &* OOH-224 %Q1bqb[199'8QORRSXS`S`a ! U]]3 %%a(=+B+B1+EEXXZ^^B' ''* ++[L#> ^^B' !4!4Q!77<)*[qj M< 99r)rrr)r r>rr>rr>r<r>) ryr>rzr>r{r>r|r>r}z Sequence[int]r<z=tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor])rr>rr>r<r>)rB __future__rcollections.abcrtypingrrtorch.nn.functionalr functionalrorfdetr.utilities.loggerrr]r __annotations__rModulerrrr__all__r rrrs  #$ . 3,87<"))7