^joddlmZddlmZddlmZddlmZmZddl Z ddl mZ ddl mZddlmZddlmZdd lmZmZmZdd lmZdd lmZmZmZdd lmZm Z dd l!m"Z"erddl#Z$GddeZ%eGddZ&y)) annotations)deepcopy) dataclass) TYPE_CHECKINGAnyN)pyplot)ORIENTED_BOX_COORDINATES) Detections) box_iou_batchmask_iou_batchoriented_box_iou_batch)LEGACY_COLOR_PALETTE)AveragingMethodMetric MetricTarget)ObjectSizeCategoryget_detection_size_category)ensure_pandas_installedcBeZdZdZej ejf ddZddZ ddZ ddZ ddZ ddZ e ddZe dd Ze dd Zdd Zdd Z dd Z ddZy)F1Scorea F1 Score is a metric used to evaluate object detection models. It is the harmonic mean of precision and recall, calculated at different IoU thresholds. In simple terms, F1 Score is a measure of a model's balance between precision and recall (accuracy and completeness), calculated as: `F1 = 2 * (precision * recall) / (precision + recall)` Examples: ```pycon >>> import numpy as np >>> import supervision as sv >>> from supervision.metrics import F1Score >>> predictions = sv.Detections( ... xyxy=np.array([[0, 0, 10, 10]]), ... class_id=np.array([0]), ... confidence=np.array([0.9]) ... ) >>> targets = sv.Detections( ... xyxy=np.array([[0, 0, 10, 10]]), ... class_id=np.array([0]) ... ) >>> f1_metric = F1Score() >>> f1_result = f1_metric.update(predictions, targets).compute() >>> round(float(f1_result.f1_50), 2) 1.0 ``` ![example_plot]( https://media.roboflow.com/supervision-docs/metrics/f1_plot_example.png ){ align=center width="800" } c<||_||_g|_g|_y)a Initialize the F1Score metric. Args: metric_target: The type of detection data to use. averaging_method: The averaging method used to compute the F1 scores. Determines how the F1 scores are aggregated across classes. N)_metric_targetaveraging_method_predictions_list _targets_list)self metric_targetrs g/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/supervision/metrics/f1_score.py__init__zF1Score.__init__Bs$, 035/1c g|_g|_y)zR Reset the metric to its initial state, clearing all stored data. N)rrrs rresetz F1Score.resetUs"$r c4t|ts|g}t|ts|g}t|t|k7r$tdt|dt|d|jj ||j j ||S)a Add new predictions and targets to the metric, but do not compute the result. Args: predictions: The predicted detections. targets: The target detections. Returns: The updated metric instance. zThe number of predictions (z) and targets (z%) during the update must be the same.) isinstancelistlen ValueErrorrextendr)r predictionstargetss rupdatezF1Score.update\s+t,&-K'4(iG { s7| +-c+.>-?@ \N*OQ  %%k2 !!'* r c,|j|j|j}|j|j|jtj \}}|j|||_|j|j|jtj\}}|j|||_|j|j|jtj\}}|j|||_ |S)z Calculate the F1 score metric based on the stored predictions and ground-truth data, at different IoU thresholds. Returns: The F1 score metric result. ) _computerr'_filter_predictions_and_targets_by_sizerSMALL small_objectsMEDIUMmedium_objectsLARGE large_objects)rresultsmall_predictions small_targetsmedium_predictionsmedium_targetslarge_predictions large_targetss rcomputezF1Score.compute{st55t7I7IJ+/+W+W  " "D$6$68J8P8P, (= $}}-> N  8 8&&(:(: N r c ~tjddd}g}t||D]\}}|j|}|j|}t |dk(rt |dkDr|j |j ^|jtjt ||jftj|j |j tjdtjft |dkDst |dk(r|jtjd|jfttjdtjtjdt|j f|jt j"k(r t%||} n_|jt j&k(r t)||} n5|jt j*k(r t-||} n t/d|j1|j |j ntj2g|j |j ntj2g| |} |j| |j |j |j f|st5|j|j6tj|j8dtjd|j8df|tj2gtddd St|D cgc]} tj:| d} } |j<| \} }}t5|j|j6| |||ddd Scc} w) aBuild per-image stats tuples and delegate to class-level computation. Each stats tuple is ``(matches, confidence, class_ids, true_class_ids)``: - Both empty: skip (no information). - Targets empty, predictions present: all predictions are FPs; true_class_ids is ``zeros((0,))``. - Targets present: IoU matching produces ``matches`` array. g?gffffff? rNdtype)rz-Unsupported metric target for IoU calculation) rr f1_scores f1_per_classiou_thresholdsmatched_classesr1r3r5)nplinspacezip_detections_contentr'class_id confidenceappendzerossizebool_int32boolfloat32intrrBOXESr MASKSr ORIENTED_BOUNDING_BOXESr r(_match_detection_batcharray F1ScoreResultrshape concatenate_compute_f1_for_classes)rpredictions_list targets_listrDstatsr*r+prediction_contentstarget_contentsioumatchesitemsconcatenated_statsrBrCunique_classess rr.zF1Score._computesOS$3$'(8,$G=  K"&":":;"G "66w?O7|q S%5%9''/;3I3I3Q  -~/B/BC288$..#,,RXX6  W!{#q(LLHHa)<)<%=TJHHT<HHT5#,, **l.@.@@+O=PQ,, 0B0BB,_>QR,, 0T0TT4+-@)K#99&//;$,,XXb\"++7 ((XXb\& GLL#'22'00#,, m= ~ "11!%!6!6((>#7#7#:;XXq.*>*>q*A&BC- "3 7"#"  EHKP5bnnUA6PP2N$2N2N 3 / <--!22%)*  Qs N:ctj| }||}||}tjtj||f}tj|d\}}tj|j dt } || tj||<|j|||| } |j| } |jtjk(rtj| d} n|jtjk(r#| jd} |j| } n|jtj k(rm| j#tj$} | jdk(r#tj| j d} ntj&| d| }  | |fS)zCompute F1 scores from concatenated stats across all images. ``unique_classes`` is the union of GT and predicted classes so that predictions of classes absent from GT still count as false positives. T) return_countsrr@axis)rjweights)rFargsortuniquer[rMrZrS searchsorted_compute_confusion_matrix _compute_f1rrMACROmeanMICROsumWEIGHTEDastyperRaverage)rrcprediction_confidenceprediction_class_idstrue_class_idssorted_indicesrf true_classes true_counts class_countsconfusion_matrixrCrBconfusion_matrix_mergeds rr\zF1Score._compute_f1_for_classess %:$:;.)3NC NNN,@A B %'IInD$Q! kxx 4 4Q 7sC FQ R__^\BC 99 )><  ''(89   O$9$9 9 15I  " "o&;&; ;&6&:&:1&= #(()@AI  " "o&>&> >'..rzz:L!Q&HH\%7%7%:; JJ|!\R ,66r c|jd|jd}}tj||ft}|dddf|k(}t |D] \}} tj || k\|z} | djds5tj | d} || dddf} tj| | g} | djddkDrh| | dddfjddd} | tj| dddfdd} | tj| dddfdd} d|| dddfjt|f<|}|S) Nrr@rkriT) return_index) rZrFrMrQ enumeratewherestackhstackrmrnrwrS)predictions_classestarget_classesrbrDnum_predictionsnum_iou_levelscorrect correct_classi iou_levelmatched_indicescombined_indices iou_valuesrcresult_corrects rrWzF1Score._match_detection_batch:s~ % %a (   #(((O^H#I.H&llnO!#!#.!9"$((>":"$''.("CQ!#.!9"$''./"J"$((>":!/!9!=!=a!@#$~h'?#?"D"DQ"G"*^";*,((/B+ Y '# *OOL    ,"4"4 4*6@oo ##++- -   ,"F"F F//%%&>?C3s8a<68hhs"**6U !!++- -243F3F2GHIIr c|jtjk(r'tjdtj }|S|jtj k(rtjdt}|S|jtjk(r'tjdtj }|Std|j)N)rr@)rrr)rrrr) rrrTrFemptyrRrUrQrVr()r empty_boxes empty_masks empty_obbs rrzF1Score._make_empty_contents   ,"4"4 43588F"**3UK    ,"4"4 413)41PK    ,"F"F F13)2::1VI 243F3F2GHIIr c~t|}|js|tjk(r|St ||j }||j k(}|j||_|j|j||_|j|j||_ |j|j||_ |j|j||_ |jG|jjD]*\}}tj|||j|<,|S)zBReturn a copy of detections with contents filtered by object size.)ris_emptyrANYrrvaluerrrJrK tracker_idrrdrFrX)rr size_categorynew_detectionssizes size_maskkeyrs r_filter_detections_by_sizez"F1Score._filter_detections_by_sizes4"*-    M5G5K5K$K! !+ND:i 0i @Pi i V57&57 7574 57 . 57   57n2-%0   >7'-7'%:7'.7', 7' ! 7'7'r1 :J$ J$5G 46*6'6* 6 3 6r rceZdZUdZded<ded<eddZeddZded <ded <d ed <d ed<ded<ded<ded<ddZddZ ddZ y)rYua The results of the F1 score metric calculation. Defaults to `0` if no detections or targets were provided. Attributes: metric_target: the type of data used for the metric - boxes, masks or oriented bounding boxes. averaging_method: the averaging method used to compute the F1 scores. Determines how the F1 scores are aggregated across classes. f1_50: the F1 score at IoU threshold of `0.5`. f1_75: the F1 score at IoU threshold of `0.75`. f1_scores: the F1 scores at each IoU threshold. Shape: `(num_iou_thresholds,)` f1_per_class: the F1 scores per class and IoU threshold. Shape: `(num_classes, num_iou_thresholds)` iou_thresholds: the IoU thresholds used in the calculations. matched_classes: the class IDs present in either predictions or ground truth. Corresponds to the rows of `f1_per_class`. Classes that appear only in predictions (no ground-truth instances) are included; their per-threshold F1 values will be `0.0`. small_objects: the F1 metric results for small objects (area < 32²). medium_objects: the F1 metric results for medium objects (32² ≤ area < 96²). large_objects: the F1 metric results for large objects (area ≥ 96²). rrrrc2t|jdS)NrfloatrBr"s rf1_50zF1ScoreResult.f1_50!T^^A&''r c2t|jdS)Nrr"s rf1_75zF1ScoreResult.f1_75%rr rrBrCrrDrrEzF1ScoreResult | Noner1r3r5c|jjd|jd|jd|jdd|j dd|j d|jd}|jjd k(r|d z }t|j|jD]\}}|d |d |d z }d }|j3|t|jjd d |z}|d|z }|j3|t|jjd d |z}|d|z }|j 3|t|j jd d |z}|d|z }|S)a, Format as a pretty string. Example: ```pycon >>> import numpy as np >>> import supervision as sv >>> from supervision.metrics import F1Score >>> predictions = sv.Detections( ... xyxy=np.array([[0, 0, 10, 10]]), ... class_id=np.array([0]), ... confidence=np.array([0.9]) ... ) >>> targets = sv.Detections( ... xyxy=np.array([[0, 0, 10, 10]]), ... class_id=np.array([0]) ... ) >>> f1_metric = F1Score() >>> f1_result = f1_metric.update(predictions, targets).compute() >>> print(f1_result) # doctest: +ELLIPSIS F1ScoreResult: Metric target: MetricTarget.BOXES Averaging method: AveragingMethod.WEIGHTED F1 @ 50: 1.0000 F1 @ 75: 1.0000 F1 @ thresh: [1. ... 1.] IoU thresh: [0.5 0.55 ... 0.95] F1 per class: 0: [1. ... 1.] ... Medium objects: F1ScoreResult: Metric target: MetricTarget.BOXES Averaging method: AveragingMethod.WEIGHTED F1 @ 50: 0.0000 ... ``` z: Metric target: z Averaging method: z F1 @ 50: z.4fz F1 @ 75: z F1 @ thresh: z IoU thresh: z F1 per class: rz No results z z:  z Small objects: z Medium objects: z Large objects: ) __class__rrrrrrBrDrCrNrHrEr1strreplacer3r5)rout_strrJ f1_of_classindentindenteds r__str__zF1ScoreResult.__str__2sR~~&&'("0012!!%!6!6 78 JJs+, JJs+, NN+, //01       ! !Q & ' 'G%()=)=t?P?P%Q 8 !Hk H:R }B7 7G 8    )D$6$6 7 ? ?6(m TTH +H:6 6G    *D$7$7 8 @ @F8} UUH ,XJ7 7G    )D$6$6 7 ? ?6(m TTH +H:6 6Gr ctddl}|j|jd}|j:|jj }|j D] \}}||d|<|j:|jj }|j D] \}}||d|<|j:|jj }|j D] \}}||d|<|j|dgS)zt Convert the result to a pandas DataFrame. Returns: The result as a DataFrame. rN)F1@50F1@75small_objects_medium_objects_large_objects_)index) rpandasrrr1 to_pandasrdr3r5 DataFrame)rpd pandas_datasmall_objects_dfrrmedium_objects_dflarge_objects_dfs rrzF1ScoreResult.to_pandasvs' !ZZZZ    )#11;;= .446 < U6; nSE23 <    * $ 3 3 = = ? /557 = U7< ocU34 =    )#11;;= .446 < U6; nSE23 <r||Ks33r cddg}|j|jg}tdgdz}|j>|j}|ddgz }||j|jgz }|tdgdzz }|j>|j}|d d gz }||j|jgz }|tdgdzz }|j >|j }|d d gz }||j|jgz }|td gdzz }dt jd<t jd\}}|jdd|jddd|jjd|jjd} |j| dtt!|} |j#| ||d} |j%| |j'|dd| D]O} | j)} |j+| j-| j/dz z| dz| ddd !Qd"t jd<t j0t j2y)#z Plot the F1 results. ![example_plot]( https://media.roboflow.com/supervision-docs/metrics/f1_plot_example.png ){ align=center width="800" } rrrrNz Small: F1@50z Small: F1@75rz Medium: F1@50z Medium: F1@75z Large: F1@50z Large: F1@75r monospacez font.family)r?)figsizerkValuebold) fontweightz"F1 Score, by Object Size (target: z , averaging: )center)coloralign-right)rotationhag{Gz?z.2fbottom)r vaz sans-serif)rrrr1r3r5pltrcParamssubplotsset_ylim set_ylabelrrr set_titleranger'bar set_xticksset_xticklabels get_heighttextget_x get_width tight_layoutshow)rlabelsvaluescolorsr1r3r5_axtitle x_positionsbarsry_values rplotzF1ScoreResult.plots7#**djj)&q)*Q.    ) ..M ~~6 6F }**M,?,?@ @F +A./!3 3F    *!00N 8 8F ~++^-A-AB BF +A./!3 3F    ) ..M ~~6 6F }**M,?,?@ @F +A./!3 3F&1 ]# W-2 Aq g& 1,,223400667q :  Uv .CK( vvk6xvH k" 6B7; Cnn&G GG cmmo11$3-   '3 ]#   r N)rr)rr)rz pd.DataFramer) rrrr__annotations__propertyrrrrr%rr rrYrYsx: %% (((('&))++**''((''BH4<>r rY)' __future__rcopyr dataclassesrtypingrrnumpyrF numpy.typingnpt matplotlibrr supervision.configr supervision.detection.corer 'supervision.detection.utils.iou_and_nmsr r r supervision.draw.colorrsupervision.metrics.corerrr%supervision.metrics.utils.object_sizerrsupervision.metrics.utils.utilsrrrrrYrr rr7sq"!%$71 8JJD^6f^6B RR Rr