^jy@$dZddlmZddlmZmZmZddlmZm Z ddl m Z er ddl Z ddl mcmZ ddd d d d ed ed edeed ff dZ dd d dd dd deedzd ed edeeefdzded eeeeezfffdZGdde ZdgZy)z"Video processor class for Videomt.)'SemanticSegmentationPostProcessorOutput)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDPILImageResampling)is_torch_availablerequires_backends)BaseVideoProcessorN mask_labels torch.Tensor mask_probs query_idxmask_thresholdoverlap_mask_area_thresholdreturnc||k(}|j}|||k\}|j}||z} | j} |dkDxr |dkDxr| dkD} | r||z } | j|kDsd} | | fS)a Checks whether a predicted query produces a valid panoptic segment. Args: mask_labels (`torch.Tensor`): Tensor of shape `(height, width)` containing the winning query index for each pixel. mask_probs (`torch.Tensor`): Tensor of shape `(num_queries, height, width)` containing per-query mask probabilities. query_idx (`int`): Index of the query to validate. mask_threshold (`float`, *optional*, defaults to 0.5): Threshold used to binarize the query mask probabilities. overlap_mask_area_threshold (`float`, *optional*, defaults to 0.8): Minimum overlap ratio required between the assigned query area and the original query mask area. Returns: `tuple[bool, torch.Tensor]`: A tuple containing whether the segment is valid and the final boolean mask for that segment. r F)sumitem) r r rrr query_maskquery_mask_area original_mask original_area final_maskfinal_mask_area mask_exists area_ratios /var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/videomt/video_processing_videomt.pycheck_segment_validityrs4 )J nn&Oy)^;M!%%'Mm+J nn&O!A%S-!*;SRS@SK$}4  #>>K  "" pred_scores pred_labelslabel_ids_to_fuse target_sizecb||jdn|d}||jdn|d}tj||ftj|jdz } g} |j }|ddddf|zj d} d} i} t|jdD]~}||j}t| ||||\}}|s+|r||vr|| vr | || |<>| | |<| | |<t||jd}| j| ||d| dz } | | fS)a Converts per-query mask predictions into a panoptic segmentation map. Args: mask_probs (`torch.Tensor`): Tensor of shape `(num_queries, height, width)` containing per-query mask logits. pred_scores (`torch.Tensor`): Tensor of shape `(num_queries,)` containing the confidence score of each predicted query. pred_labels (`torch.Tensor`): Tensor of shape `(num_queries,)` containing the predicted class ID of each query. label_ids_to_fuse (`set[int]`, *optional*): Label IDs that should be fused across disconnected regions. mask_threshold (`float`, *optional*, defaults to 0.5): Threshold used to binarize the query mask probabilities. overlap_mask_area_threshold (`float`, *optional*, defaults to 0.8): Minimum overlap ratio required to keep a predicted segment. target_size (`tuple[int, int]`, *optional*): Final `(height, width)` of the segmentation map. If unset, uses the spatial size of `mask_probs`. Returns: `tuple[torch.Tensor, list[dict[str, int | float]]]`: The panoptic segmentation map and the metadata for each predicted segment. Nr )dtypedeviceidlabel_idscore) shapetorchzeroslongr(sigmoidargmaxrangerrroundappend)r r r!r"rrr#heightwidth segmentationsegmentsr current_segment_idstuff_memory_listr pred_classrr segment_scores rcompute_segmentsr?Hs~@%0$7Z  a [^F#.#6J  Q KNE;;ejjIZIZ[^__LH##%Jq$}- :BB1EK(*;,,Q/0  +002 "8 Y@[#  Z  /@!@..+&E,E thresholdc t|dg|j}|j}|j||}|j}|j d}|j d} g} t |D]} || } || } | jjddddf}|jd\}}| dkD}| jj}|jd|jdzjd|jdjdd zz }||z}tj|| dtj| }g}d}t | D]q}||j!}tj"||s/||k\s5||||<|j%|||j!t'|d d |dz }s| j%||d | S)a Converts the output of [`VideomtForUniversalSegmentation`] into instance segmentation predictions. Args: outputs ([`VideomtForUniversalSegmentationOutput`]): Raw outputs of the model. target_sizes (`list[tuple[int, int]]`): List of `(height, width)` tuples corresponding to the requested final size of each prediction. Length should match the number of frames in the output. threshold (`float`, *optional*, defaults to 0.5): Minimum combined score to keep an instance. Returns: `list[dict]`: A list of dicts (one per frame), each containing: - `"segmentation"` -- A `torch.Tensor` of shape `(height, width)` with instance IDs (or -1 for background). - `"segments_info"` -- A list of dicts with `"id"`, `"label_id"`, and `"score"` for each instance. r/r rXrUrV.Nr%gư> fill_valuer'r(r)r*r9 segments_info)rr[rCrRr(r.r4r\r]maxr2r`rr/fullr1ranyr6r5)rMrbrDrjr[rCmask_probs_batchr( num_frames num_queriesresults frame_idx mask_pred mask_class class_probsscores pred_classes pred_masksr mask_scoresr r9r:r;rr-s r"post_process_instance_segmentationz8VideomtVideoProcessor.post_process_instance_segmentations. $ *&;;&;;334H,W%,,)//2 *004 z* VI(3I-i8J$**,444rs,)OZZ:8## ), *#*#*#*# *# "' *#  4  *#d ),*.G"G"G" G"3x$ G"  G" "' G"sCx4'G" >4S#+%5 67 78G"Tk.k\ # #r