^jV VdZddlZddlZddlmZmZddlZddlZddl m Z ddl m Z ddlmZddlmZmZdd lmZmZmZmZmZmZdd lmZmZdd lmZmZm Z m!Z!m"Z"e rdd l#m$Z$dd l%m&Z&m'Z'erddl(m)Z)e!jTe+Z,GddedZ- d/de.e/zej`zde1de1de2de2f dZ3de2dej`dej`dej`fdZ4dZ5d ej`d!e.ej`ej`ffd"Z6d0d#ej`d$ej`d%e1d!ej`fd&Z7d#ej`d'ej`d(ej`d)ej`d!ej`f d*Z8d+ej`d!ej`fd,Z9eGd-d.e Z:d.gZ;y)1z"Image processor class for VitPose.N) TYPE_CHECKINGUnion) functional)TorchvisionBackend) BatchFeature)group_images_by_shapereorder_images)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDChannelDimension ImageInputPILImageResamplingSizeDict) ImagesKwargsUnpack) TensorTypeauto_docstringis_scipy_availableloggingrequires_backends)inv)affine_transformgaussian_filter)VitPoseEstimatorOutputc2eZdZUdZedzed<edzed<y)VitPoseImageProcessorKwargsaI do_affine_transform (`bool`, *optional*): Whether to apply an affine transformation to the input images based on the bounding boxes. normalize_factor (`float`, *optional*, defaults to `200.0`): Width and height scale factor used for normalization when computing center and scale from bounding boxes. Ndo_affine_transformnormalize_factor)__name__ __module__ __qualname____doc__bool__annotations__float/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/vitpose/image_processing_vitpose.pyrr7s$dl"r)rF)totalbox image_width image_heightr padding_factorc.|dd\}}}}||z } tj||dzz||dzzgtj} || |zkDr |dz| z }n || |zkr|| z}tj||z ||z gtj} | |z} | | fS)a Encodes a bounding box in COCO format into (center, scale). Args: box (`Tuple`, `List`, or `np.ndarray`): Bounding box in COCO format (top_left_x, top_left_y, width, height). image_width (`int`): Image width. image_height (`int`): Image height. normalize_factor (`float`): Width and height scale factor. padding_factor (`float`): Bounding box padding factor. Returns: tuple: A tuple containing center and scale. - `np.ndarray` [float32](2,): Center of the bbox (x, y). - `np.ndarray` [float32](2,): Scale of the bbox width & height. N?dtype?)nparrayfloat32) r,r-r.r r/ top_left_x top_left_ywidthheight aspect_ratiocenterscales r*box_to_center_and_scaler@Ds:-0G)J E6-L XXzECK/fsl1JKSUS]S] ^F |f$$|+ & &% HHe..9I0IJRTR\R\ ]E N "E 5=r)theta size_inputsize_dst size_targetctj|}tjdtj}|d|dz }|d|dz }t j ||z|d<t j | |z|d<|d|dzt j |zd|dzt j |zzd|dzzz|d <t j ||z|d <t j ||z|d <|d|dzt j |zd|dzt j |zz d|dzzz|d <|S) a Calculate the transformation matrix under the constraint of unbiased. Paper ref: Huang et al. The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (CVPR 2020). Source: https://github.com/open-mmlab/mmpose/blob/master/mmpose/core/post_processing/post_transforms.py Args: theta (`float`): Rotation angle in degrees. size_input (`np.ndarray`): Size of input image [width, height]. size_dst (`np.ndarray`): Size of output image [width, height]. size_target (`np.ndarray`): Size of ROI in input plane [w, h]. Returns: `np.ndarray`: A matrix for transformation. )rr3rrrrrrgr2rrFrrrrrrF)r6deg2radzerosr8mathcossin)rArBrCrDmatrixscale_xscale_ys r*get_warp_matrixrUpsX( JJu E XXfBJJ /FqkKN*GqkKN*G88E?W,F4LHHUO#g-F4L z!}txx.z!}1DtxxPU1VVY\_jkl_mYmmF4L88E?W,F4L88E?W,F4L z!}txx.z!}1DtxxPU1VVY\_jkl_mYmmF4L Mr)c t|jdDcgc] }|d|f }}tj|gdg}t |}|d|d|d|d|d|d f\|d<|d<|d<|d<|d <|d<|Dcgc]}t |||d }}tj |d }|Scc}wcc}w) a[ This function implements cv2.warpAffine function using affine_transform in scipy. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.affine_transform.html and https://docs.opencv.org/4.x/d4/d61/tutorial_warp_affine.html for more details. Note: the original implementation of cv2.warpAffine uses cv2.INTER_LINEAR. .)rrrrKrJrHrGrLrIr) output_shapeorderaxis)rangeshaper6vstackrrstack) srcMsizeichannelsM_scipyM_invchannelnew_srcs r*scipy_warp_affineris &+399R=%9:CF :H:iiI'G LE d  d  d  d  d  d TPE$KteDk5;d U4[\ddPWTKdGdhhwR(G N#;es B7B<heatmapsreturnct|tjs td|jdk7r t d|j \}}}}|j||df}tj|dj||df}tj|dj||df}tj|djtj}|ddddd f|z|ddddd f<|dddddf|z|dddddf<tjtj|dd kD|d}||fS) aGet keypoint predictions from score maps. Args: heatmaps (`np.ndarray` of shape `(batch_size, num_keypoints, height, width)`): Model predicted heatmaps. Returns: tuple: A tuple containing aggregated results. - coords (`np.ndarray` of shape `(batch_size, num_keypoints, 2)`): Predicted keypoint location. - scores (`np.ndarray` of shape `(batch_size, num_keypoints, 1)`): Scores (confidence) of the keypoints. zHeatmaps should be np.ndarrayr1z Heatmaps should be 4-dimensionalrWrFr)rrrFNrg) isinstancer6ndarray TypeErrorndim ValueErrorr]reshapeargmaxamaxtileastyper8where) rj batch_size num_keypoints_r;heatmaps_reshapedidxscorespredss r*get_keypoint_predictionsrs- h +788}};<<*2..'J q% ((*mR)HI ))%q ) 1 1:}a2P QC WW& * 2 2J q3Q RF GGC # * *2:: 6E1a7^e+E!Q'N1a7^u,E!Q'N HHRWWVY/#5ub AE &=r)coordsbatch_heatmapskernelc|j\}}}}|jd}|dk(s||k(s tdt|dz dz}tj|D cgc]!} | D cgc]} t | d||fdc} #c} } }tj |dd }tj|}tj|d d j} |d dz|ddz|dzzz} | |dz|dzztjd||zjd|zz } | jtjdd} | | } | | dz}| | |zdz}| | |zdz}| | |z dz }| | dz }| | dz |z }d||z z}d||z z}tj||gd}|j||dd}|d| zz |z}|d| zz |z}d||z |z | z| z|z |z |zz}tj||||gd}|j||dd}tjj|tj tj"j$tj&dzz}|tj(d||j+z}|Scc} wcc} } w)aDARK post-processing. Implemented by unbiased_data_processing. Paper references: - Huang et al. The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (CVPR 2020). - Zhang et al. Distribution-Aware Coordinate Representation for Human Pose Estimation (CVPR 2020). Args: coords (`np.ndarray` of shape `(num_persons, num_keypoints, 2)`): Initial coordinates of human pose. batch_heatmaps (`np.ndarray` of shape `(batch_size, num_keypoints, height, width)`): Batched heatmaps as predicted by the model. A batch_size of 1 is used for the bottom up paradigm where all persons share the same heatmap. A batch_size of `num_persons` is used for the top down paradigm where each person has its own heatmaps. kernel (`int`, *optional*, defaults to 3): Gaussian kernel size (K) for modulation. Returns: `np.ndarray` of shape `(num_persons, num_keypoints, 2)` ): Refined coordinates. rrzQThe batch size of heatmaps should be 1 or equal to the batch size of coordinates.rFg?rH)sigmaradiusaxesgMbP?2)rGrGrKrKedge)mode).r).rrWrr2rZzijmn,ijnk->ijmk)r]rqintr6r7rcliplogpadflattenarangerrrv concatenatelinalgrfinfor8epseyeeinsumsqueeze)rrrrxryr<r; num_coordsrrjheatmapbatch_heatmaps_padindexi_ix1iy1ix1y1ix1_y1_ix1_iy1_dxdy derivativedxxdyydxyhessians r*"post_dark_unbiased_data_processingrs*0>/C/C,J vuaJ !OzZ7lmm &1*" #FXX+ go o[b_WC8Hv V o N WW^UB7NVVN+N0PW]^ffh 6NQ &.1"4!C CE eaiFQJ '"))AzM7Q*R*Z*Z[]_l*m mmE LL  % %b! ,E E "B UQY 'C UU]Q. /C uu}q0 1E !23G eai (D eai%/ 0D d B d BRq1J##J q!DJ B, C B, C s"R'",t3d:WD ECnnc3S1:Gooj-A>GiimmGbhhrzz&:&>&>&JJKG bii)7J?GGIIF M; p s J> $J9<J> 9J> r>r? output_sizec|jddvr tdt|dk7r tdt|dk7r tdt|dk7r td|dz}|d|d d z z }|d |dd z z }tj|}|d d d f|z|d z|d d zz |d d d f<|d d df|z|dz|dd zz |d d df<|S) aoGet final keypoint predictions from heatmaps and apply scaling and translation to map them back to the image. Note: num_keypoints: K Args: coords (`np.ndarray` of shape `(num_keypoints, ndims)`): * If ndims=2, corrds are predicted keypoint location. * If ndims=4, corrds are composed of (x, y, scores, tags) * If ndims=5, corrds are composed of (x, y, scores, tags, flipped_tags) center (`np.ndarray` of shape `(2,)`): Center of the bounding box (x, y). scale (`np.ndarray` of shape `(2,)`): Scale of the bounding box wrt original image of width and height. output_size (`np.ndarray` of shape `(2,)`): Size of the destination heatmaps in (height, width) format. Returns: np.ndarray: Predicted coordinates in the images. r)rFr1z5Coordinates need to have either 2, 4 or 5 dimensions.rFz9Center needs to have 2 elements, one for x and one for y.z,Scale needs to consist of a width and heightz2Output size needs to consist of a height and widthi@rr5Nr2)r]rqlenr6 ones_like)rr>r?rrTrS target_coordss r*transform_predsr s2||Ai'PQQ 6{aTUU 5zQGHH ;1MNN EMEAh+a.3./GAh+a.3./GLL(M A,06!9r?rotationrbct||dztj|j|jfdz |dz}|j dddj j}t|||j|jf}tj|j dddj|j} | S)z7Apply an affine transformation to a torch tensor image.g@r5rrrFr)r`rarb) rUr6r7r;r<permutecpunumpyritorch from_numpytor) rrr>r?rrbtransformationimage_nptransformed_np transformeds r*rz&VitPoseImageProcessor.affine_transforms) fslBHHdjj$++-F$G#$MuW\} ==Aq)--/557*x>QUQ\Q\^b^h^hPij&&~6>>q!QGJJ5<<X r) do_resizeresamplez7PILImageResampling | tvF.InterpolationMode | int | Nonedo_center_crop crop_size do_rescalerescale_factor do_normalize image_mean image_stddo_padpad_sizedisable_groupingreturn_tensorsrr c |n|rlg}t||D]Y\}}|D]O}t||j|j|\}}|j |||d|}|j |Q[|}t ||\}}i}|jD] \}}|j|||| | | }|||<"t||} td| i|S)z!Custom preprocessing for VitPose.)r-r.r r)rrb)rr)data tensor_type) zipr@r;r<rappendr itemsrescale_and_normalizer r)!rrrrbrrrrrrrrrrrrrr rrtransformed_imagesr image_boxesr,r>r?transformed_imagegrouped_imagesgrouped_images_indexprocessed_images_groupedr]stacked_imagesprocessed_imagess! r*rz!VitPoseImageProcessor._preprocesss.  !4!# &)&%&8 A"{&AC$;$(JJ%)[[)9 %MFE )-(=(=eVU]^ei(=(j%&--.?@A A(F/DV^n/o,,#% %3%9%9%; = !E>!77 NL*V_N/= $U +  = **BDXY.2B!CQ_``r)rjrc|j\}}}}t|\} } t| ||} t|D]} t | | || || ||g| | <!| | fS)zRGet final keypoint predictions from heatmaps and transform them back to the image.r)r>r?r)r]rrr\r) rrjr>r?rrxrzr<r;rr}r~rcs r*keypoints_from_heatmapsz-VitPoseImageProcessor.keypoints_from_heatmapssy(0~~$ Avu1(;268FSz" pA&uQxq q`fhm_noE!H pf}r)outputsr kernel_size threshold target_sizescDt|d|jj\}}}}||t|k7r t dt j |dft j} t j |dft j} ttj|} t|D]|} |4|| d|| d}} t j| || |g}| | |z| | <|jd|jd }}t| | || \}}|| | ddf<|| | ddf<~|j|jj!j#| | | \}}t j |d ft j}| ddddf|ddddf<| ddddf|dddd f<t%j&|}t%j&|}t%j(d|}t%j&t+|}g}t-|||}|D]k}g}|D]Q}t/|\}}} |j1}|}!|||kD}"||"}||"}|!|"}!|||!| d }#|j3|#S|j3|m|S)a Transform the heatmaps into keypoint predictions and transform them back to the image. Args: outputs (`VitPoseEstimatorOutput`): VitPoseForPoseEstimation model outputs. boxes (`list[list[list[float]]]` or `np.ndarray`): List or array of bounding boxes for each image. Each box should be a list of 4 floats representing the bounding box coordinates in COCO format (top_left_x, top_left_y, width, height). kernel_size (`int`, *optional*, defaults to 11): Gaussian kernel size (K) for modulation. threshold (`float`, *optional*, defaults to None): Score threshold to keep object detection predictions. target_sizes (`torch.Tensor` or `list[tuple[int, int]]`, *optional*): Tensor of shape `(batch_size, 2)` or list of tuples (`tuple[int, int]`) containing the target size `(height, width)` of each image in the batch. If unset, predictions will be resize with the default value. Returns: `list[list[Dict]]`: A list of dictionaries, each dictionary containing the keypoints and boxes for an image in the batch as predicted by the model. rNzTMake sure that you pass in as many target sizes as the batch dimension of the logitsrFr3rrr;r<)r-r.rr1) keypointsr}labelsbbox)rrjr]rrqr6rNr8list itertoolschainr\r7rbr@rrrrtensorrrrnextrr)$rrrrrrrxryrzcentersscalesflattened_boxesrcr-r. scale_factorr;r<r>r?r~r} all_boxesposesr bboxes_xyxyresultspose_bbox_pairs image_bboxes image_resultsposescore bbox_xyxykeypoints_labelskeep pose_results$ r*post_process_pose_estimationz2VitPoseImageProcessor.post_process_pose_estimations8 $(*1*:*:*@*@' M1a  # c,6G(Gst t((J?"**=:q/<y67z" !A',8OA,> QPQ@R\ !xxlKQ](^_ %4Q%7,%F" IIg. (0C6E3OA4FTYhnoMFE"GAqDM F1a4L !44    " ( ( *GVK5 vHHj!_BJJ? #AqsFO !QqS&"1ac6N !QqS& U#f%a/ll#5i#@A 79eV[9! *L;=M! 2)-o)>&eY #) ( 9,D:D!$KE'7'=$,0EM]gpq $$[1 2 NN= ) *r))N)TrN) )rNN)+r!r"r#r$r valid_kwargsmodel_input_namesr rr rrbrrrr rrrrrr'r6rnrrr%r rstrrrcompilerdisabletuplerrrrrrr __classcell__)rs@r*rrPsI@.L'(&J$IC (DJL#(C!D# ; ;De%&3 ;45 ;  ; ;&59 22De%&3d:2 2 , 2 c>)*T1 2 2" ^^e U|      D%)"'*.'-a^$-a-a -a L -a  -a-a-a-a-aDK'$.-a4;&--at -aT/-a+-a j(4/!-a""#-a$ %-a&bjj 4''-a* +-ah **   zz   &"&8< F)FDe%&3F F 4< F !4;.5 Fr)r)rg?)r)r-s) ' 7;2E5 ?8   H % #,e #"$ )  "))) )  )X"5"bjj"BJJ"]_]g]g"J4rzzeBJJ