^jBdZddlZddlZddlZddlmZmZmZmZm Z m Z ddl Z ddl m Z ddlmZddlmZddlmZdd lmZmZmZ d'd e eefd e ee eeffd ed edeedede ee eefffdZededfZGdde j:j<ZGdde j:j<Z Gdde j:j<Z!Gdde j:j<Z"d(dZ#Gdde j:j<Z$ d)de jJd e eefd ed!ed"ed#ede e jJe e jJe jJe jJfffd$Z&Gd%d&e j:j<Z'y)*az NaFlex (NaViT + FlexiViT) Transforms and Collation Implements PyTorch versions of the transforms described in the NaViT and FlexiViT papers: - NaViT: https://arxiv.org/abs/2307.14995 - FlexiViT: https://arxiv.org/abs/2212.08013 Enables variable resolution/aspect ratio image handling with efficient patching. Hacked together by / Copyright 2025, Ross Wightman, Hugging Face N)DictListOptionalSequenceTupleUnion)Image) transforms) functional)InterpolationMode)str_to_interp_mode crop_or_padcenter_crop_or_padimage_hw patch_size max_seq_lendivisible_by_patch max_ratioepsreturnc t|tr||c nt|dk7r td|\ dks dkr td fd fd}|dz }d}||z |k\r||zd z } || r| }n| }||z |k\r|} | t | |} | |kr td | dk\r td | } | | fS) aDetermine scaling ratio and image size for sequence length constraint. Calculates the scaling ratio needed so that when image_hw is scaled, the total number of resulting patches does not exceed max_seq_len. Args: image_hw: Original image dimensions (height, width). patch_size: Patch dimensions. If int, patches are square. max_seq_len: Maximum allowed sequence length. divisible_by_patch: Whether resulting dimensions must be divisible by patch_size. max_ratio: Optional cap on scaling ratio to prevent excessive upsampling. eps: Convergence threshold for binary search. Returns: Tuple of (ratio, target_hw) where ratio is the scaling factor and target_hw is the resulting (height, width) after scaling. Cpatch_size tuple must have exactly two elements (patch_h, patch_w).rz'patch_size dimensions must be positive.cd|z}d|z}r6tj|z z}tj|z z}tt|}tt|}||fS)zYScale image_hw by ratio and optionally round dimensions to multiples of patch_h, patch_w.rr )mathceilintmax)ratioscaled_hscaled_wrrpatch_hpatch_ws f/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/data/naflex_transforms.pyprepare_target_hwz1get_image_size_for_seq..prepare_target_hwAsA;&A;& 8g+=!>>H8g+=!>>Hs8W-.s8W-.!!cB |\}}|z}|z}||z}|kS)zACheck if scaling by 'ratio' keeps patch count within max_seq_len.) r t_ht_w num_patches_h num_patches_wseq_lenrr#r$r&s r% is_feasiblez+get_image_size_for_seq..is_feasibleQs:$U+Sw w -/+%%r'g$@gY@g@z0Binary search failed - image might be too large?z0Binary search failed - image might be too small?) isinstancerlen ValueErrormin)rrrrrrr/lbrbmidr target_hwr#r$r&s` `` @@@r%get_image_size_for_seqr8s 6*c"%z z?a bc c%!|w!|BCC" & tB B 7s Bw#o s BB 7s  EE9% |KLL ~KLL"%(I ) r'bilinearbicubicceZdZdZ d dedededeedee e e e dffd df fd Z d e jd e jfd ZxZS)ResizeToSequencezResize image to fit within a maximum sequence length constraint when patchified. This maintains aspect ratio while ensuring the resulting image, when divided into patches, will not exceed the specified maximum sequence length. Nrrrr interpolation.rct|||_||_||_||_t |tr"|dk(r t|_ yt||_ y||_ y)ajInitialize ResizeToSequence transform. Args: patch_size: Size of patches. max_seq_len: Maximum sequence length constraint. divisible_by_patch: Whether dimensions must be divisible by patch_size. max_ratio: Optional cap on scaling ratio. interpolation: Interpolation method or methods. randomN) super__init__rrrrr0str_RANDOM_INTERPOLATIONr=r)selfrrrrr= __class__s r%rAzResizeToSequence.__init__s]" $&"4" mS )(%:"%7 %F"!.D r'imgctjj|\}}}t||f|j|j |j |j\}}t|jttfr tj|j}n |j}tjj|||d}|S)zResize image to maintain aspect ratio and fit sequence constraint. Args: img: Input image tensor. Returns: Resized image tensor. )rrT)r= antialias)r r get_dimensionsr8rrrrr0r=tuplelistr?choiceresize)rDrF_hwr7r= resized_imgs r%forwardzResizeToSequence.forwards''66s;1a- F OO   #66nn  9 d((5$- 8"MM$*<*<=M ..M ++223 Q^jn2o r')TNr:)__name__ __module__ __qualname____doc__rboolrfloatrrBr rrAtorchTensorrR __classcell__rEs@r%r<r<~s $'+)-Zc ///!% /   / !&7?PRU?U9V!VW //>5<<ELLr'r<cdeZdZdZ dfd Ze ddZdZdZxZ S)ResizeKeepRatioToSequencezS Resize and Keep Aspect Ratio, adapted to fit sequence length constraints. c t |||_||_||_t ||_|dk(r t|_nt||_||_ ||_ ||_ | |_ | |_| |_y)ua Args: patch_size: Size of patches (int or tuple of (patch_h, patch_w)) max_sequence_len: Maximum allowed sequence length for the resulting image divisible_by_patch: If True, ensure dimensions are divisible by patch_size longest: Float between 0-1 where 0=shortest side, 1=longest side determines scale interpolation: Interpolation method for resizing random_scale_prob: Probability of applying random scaling random_scale_range: Range for random scaling factor (min, max) random_scale_area: If True, scale factors affect area (√ factor) random_aspect_prob: Probability of applying random aspect ratio jittering random_aspect_range: Range for random aspect ratio (min, max) max_ratio: Maximum allowed scaling ratio r?N)r@rArmax_sequence_lenrrYlongestrCr=rrandom_scale_probrandom_scale_rangerandom_scale_arearandom_aspect_probrandom_aspect_ranger) rDrrarrbr=rcrdrerfrgrrEs r%rAz"ResizeKeepRatioToSequence.__init__s~8 $ 0"4W~ H $!6D !3M!BD !2"4!2"4#6 "r'c Ntj|dd\} } t| | f|||| \} }|\}}|| z }|| z }t|||zt ||d|z zz}|dkDrRt j |kr;t j |d|d}|rdtj|z }||f}nd}|dkDrt j |krztj| dtj| df}tjt j |}tj|}|d|z |d|zf}t| | f|Dcgc]\}}t||z|z}}}t|tr||}}n|\}}t|d||d<t|d||d<|rB|tj|d|z z|d<|tj|d|z z|d<|d|z}|d|z}||z}||kDrtj||z }t|d|z|d<t|d|z|d<|rB|tj|d|z z|d<|tj|d|z z|d<|Scc}}w)zGet parameters for resizing.r N?r)riri)FrIr8rr3r?uniformrsqrtlogexpziproundr0rr) rFrrarrbrcrdrerfrgrimg_himg_wrNr7target_htarget_wratio_hratio_wr ratio_factor log_aspect aspect_factordimfsizephpwr,r-r. scale_backs r% get_paramsz$ResizeKeepRatioToSequence.get_paramss '',QR0 u. EN      9'(U"U"GW%/#gw2G2PW<2XX q V]]_7H%H!>>*J$q'J./DG$q'J./DG"tyya266Qtyya266Q CXs!J!c |j||j|j|j|j|j |j |j|j|j|j }t|jttfr tj |j}n |j}t#j$|||S)zb Resize the image with aspect ratio preservation and sequence length constraints. )rrrarrbrcrdrerfrgrr0r=rJrKr?rLrjrM)rDrFr|r=s r%rRz!ResizeKeepRatioToSequence.forwardKs  OO  ! !  # # LL  " "  # #  " "  # #  $ $ NN   d((5$- 8"MM$*<*<=M ..MxxT=11r'c *t|jttfrdnt |j}|j j d|jd|jd|jdd|jdd|jdd S) Nr? (patch_size=, max_sequence_len=z , longest=z.3fz, random_scale_prob=z, random_aspect_prob=)) r0r=rJrKrBrErTrrarbrcrf)rDinterpolate_strs r%__repr__z"ResizeKeepRatioToSequence.__repr__ds&01C1CeT]&S(Y\]a]o]oYp>>**+<7HI$$($9$9#:;<<,-%%)%;%;C$@A&&*&=&=c%B! E Fr') rSTr9r)g333333?g?Frg?g(\?N)r)rigHzG?FrrN) rTrUrVrWrA staticmethodrrRrr\r]s@r%r_r_sg !#$ +#! +,#\!*#! +UUn22Fr'r_c\eZdZdZ d dedededeeeeeeffdef fd Z dZ xZ S) CenterCropToSequencezVCenter crop the image such that the resulting patch sequence length meets constraints.rrrfill padding_modecht|||_||_||_||_||_yN)r@rArrrrr)rDrrrrrrEs r%rAzCenterCropToSequence.__init__os6 $&"4 (r'ctjj|\}}}t||f|j|j |j \}}t|||j|jS)zKCenter crop the image to maintain aspect ratio and fit sequence constraint.)rr) r r rIr8rrrrrr)rDrFrNrOrPr7s r%rRzCenterCropToSequence.forwardsi''66s;1a- F OO     # #  9"#ytyytO`O`aar'Trconstant) rTrUrVrWrrXrrrBrArRr\r]s@r%rrmsa` (,56 * ) ) )!% ) U3S=112 )  ) br'rcxeZdZdZ d dedededeeeeeeffdef fd Z e dZ d Z d efd Z xZS) RandomCropToSequencea,Randomly crop and/or pad the image to fit sequence length constraints. This maintains aspect ratio while ensuring the resulting image, when divided into patches, will not exceed the specified maximum sequence length. Similar to CentralCropToSequence but with randomized positioning. rrarrrcht|||_||_||_||_||_y)a Args: patch_size: Size of patches (int or tuple of (patch_h, patch_w)) max_sequence_len: Maximum allowed sequence length for the resulting image divisible_by_patch: If True, resulting image dimensions will be multiples of patch_size fill: Fill value for padding padding_mode: Padding mode ('constant', 'edge', 'reflect', 'symmetric') N)r@rArrarrr)rDrrarrrrEs r%rAzRandomCropToSequence.__init__s6 $ 0"4 (r'c tjj|\}}}||dz }||dz }|dk(rd}n>sC5%. EN OO  ! !  # #  9OOC3 T Q<A,**  r'rc|jjd|jd|jd|jdS)Nrr, divisible_by_patch=r)rErTrrar)rDs r%rzRandomCropToSequence.__repr__sN>>**+<7HI$$($9$9#:;&&*&=&=%>aA Br'r)rTrUrVrWrrXrrrBrArrrRrr\r]s@r%rrs(,56 * ))")!% ) U3S=112 )  ).& :B#Br'rct|trt||k7rt|d|d|d|dkDr0t j |j d|d|dfS|S)Nz should be a sequence of length .rr z range reversed. Swapping.)r0rr1r2warningswarn capitalize)valuenamelengths r%_validate_rangersv eX &#e**>D6!A&KLL Qx%( *++EFGQxq!! Lr'ceZdZdZ ddeeeeeffded eeefd eeefd eee fd e d e ede eeefdeffd Z e dddddddefdej d eeefd eeefdedededed e d e ede eeefd eee e fdeeeeeefeeefe ffdZdej dej fdZdefdZxZS)RandomResizedCropToSequencea Randomly crop the input image to a subregion with varying area and aspect ratio (relative to the original), then resize that crop to a target size. The target size is determined such that patchifying the resized image (with `patch_size`) does not exceed `max_seq_len` patches, while maintaining the aspect ratio of the crop. This combines aspects of torchvision's RandomResizedCrop with sequence length constraints. Args: patch_size (int or tuple[int, int]): Patch dimensions (patch_h, patch_w) for sequence length calculation. max_seq_len (int): Maximum number of patches allowed in the final image. scale (tuple[float, float]): Range (min, max) of area fraction of the original image to crop. ratio (tuple[float, float]): Range (min, max) of aspect ratio *multipliers* for the crop, relative to the original image's aspect ratio. E.g., (0.75, 1.333) means the crop's aspect ratio will be sampled between 0.75*orig_ar and 1.333*orig_ar. Uses log-uniform sampling. interpolation (str or InterpolationMode): Interpolation mode for resizing. Can be 'bilinear', 'bicubic', 'nearest', or 'random' (chooses between bilinear and bicubic). Defaults to 'bicubic'. divisible_by_patch (bool): If True, the final image height and width will be multiples of the respective patch dimensions. Defaults to True. max_ratio (float, optional): An optional upper limit on the scaling ratio applied during resizing. Prevents excessive upsampling of the initial crop. `max_ratio=1.0` prevents any upsampling beyond the cropped size. Defaults to None (no limit). final_scale_range (tuple[float, float], optional): If provided, applies an *additional* random scaling factor to the final target size. The factor is sampled uniformly from this range, and multiplied by the size determined by `get_image_size_for_seq`. E.g., (0.8, 1.0) means the final size will be between 80% and 100% of the maximum feasible size. Defaults to None (use maximum feasible size). attempts (int): Number of attempts to sample a valid crop geometry before falling back to a center crop strategy. Defaults to 10. rrSTN rrscaler r=rrfinal_scale_rangeattemptsc t |t|tr||c|_|_n(t |dk7r td|\|_|_||_||_ ||_ ||_ ||_ ||_ | |_t|tr"|dk(r t |_nt%||_n||_t'|jd|_ t'|jd|_ |j_t'|jd|_ d|jdcxkr|jd cxkrd ksnt)j*d yyy) Nrrr?rr rrrr riz?final_scale_range values should ideally be between 0.0 and 1.0.)r@rAr0rr#r$r1r2rrr rrrrrBrCr=rrrr) rDrrrr r=rrrrrEs r%rAz$RandomResizedCropToSequence.__init__sB  j# &)3Z &DL$,:!# !fgg)3 &DL$,&  "4"!2  mS )(%:"%7 %F"!.D %TZZ9 $TZZ9   ! ! -%4T5K5KM`%aD "411!4X8N8Nq8QXUXX _`Y .r'rF crop_attemptsr#r$rc tj|\} } } | dks| dkrtd| d| | | z}| | z }tj|dtj|df}t |D]} |t j|d|dz}tjt j|d|d}||z}tttj||z }tttj||z}d|cxkr| ksnd|cxkr| ksnt jd| |z }t jd| |z }n||dz}||dz}||kr$| }ttt||z | }n-||kDr$| }ttt||z| }n| }| }td|}td|}| |z dz}| |z dz}t||f||f|||\}}|}| | \}}t j||}tt|dd}||z|z} ||z|z}!|r7|tj | |z z}"|tj |!|z z}#n(tt| }"tt|!}#t|"|}"t|#|}#|"|#f}|d|z}$|d|z}%|$|%z|kDr%|}t#j$d |d d |d |d |d t'| t(t*frt j,| } n| } ||||f|| fS)zP Get parameters for a random sized crop relative to image aspect ratio. rz1Input image must have positive dimensions, got H=z, W=r r)rrrrrrizFinal scale randomization (z.2fz) resulted in size z exceeding max_seq_len=z, after rounding. Reverting to feasible size r)rjrIr2rrmranger?rkrnrrprlrr3rr8rrrr0rJrKrL)&rFrr rr#r$rrrrr=rNrrarea orig_aspect log_ratio target_areaaspect_ratio_factor aspect_ratiocrop_hcrop_wrrmin_aspect_ratiomax_aspect_ratiofeasible_ratio feasible_size final_sizemin_scmax_sc scale_factorraw_hraw_wrsrtr,r-s& r%rz&RandomResizedCropToSequence.get_paramsJs ++C065 Q;%1* QRXQYY]^c]def f~fn XXeAh'%();< }%' )Aa%(!CCK"&((6>>)A, RS +U"V &)<>&&9Ls<5s;L^+l:E^+l:E""TYYuw%??"TYYuw%??uU|,uU|,8W-H8W-H"H-J'qMW4M&qMW4M -<+!<\#z7RandomResizedCropToSequence.__repr__..s"'Z!A S(9"(='Zs')(z patch_size=(rz, max_seq_len=z, scale=z, ratio=z, interpolation=[]rz , max_ratio=z, final_scale_range=z , attempts=)r0r=rJrKjoinrBrErTr#r$rrr rrrr)rDr format_strings r%rz$RandomResizedCropToSequence.__repr__s9 d((5$- 8"ii'ZtGYGY'ZZO!$"4"45O//#5 < ~R ~QGG >$*:*:);<< 8DJJ<00 8DJJ<00 ,_,=Q?? 01H1H0IJJ <'788 /0F0F/GHH ;t}}o66  r') rrS)g{Gz?ri)g?g?r:TNNr)rTrUrVrWrrrrYrBr rXrrArrCrZr[rrrRrr\r]s@r%rrs(X35%0%/7@#'%);?-a#uS#X./-a-aUE\" -a UE\" -a S"334 -a!-aE?-a$E%,$78-a-a^ "$#'+)-?COdrFrF&rF&rF rF  rF  rFrF!%rF rF (eUl(;<rF!&7!8:K!KLrF uS#sC'(%S/;LL MrFrFh5<<ELL:#r'rrFpad include_infoflatten_patches channels_lastc|j\}}}|\} } |rd|| zdk7s|| zdk7rT| || zz | z} | || zz | z} tjjj |d| d| f}|j\}}}|| z|| z}} |rW|j || | || j ddddd}|r|jd| | z|zn|jd| | |}nV|j || | || j ddddd}|r|jd|| z| zn|jd|| | }|rtjtj| tj|d\}}tj|jd|jdgd }tj| |ztj }|||fS|S) aPatchify a single ``(C, H, W)`` image. Args: img: Input image tensor of shape ``[C, H, W]``. patch_size: Patch dimensions ``(patch_h, patch_w)``. pad: Whether to zero-pad to a multiple of patch size. include_info: Also return ``(patch_coord, patch_valid)``. flatten_patches: If ``True``, returned patches are 2-D ``[N, P*P*C]``; otherwise 4-D (unflattened per-patch). channels_last: Per-patch flat layout. ``True`` (default) yields ``P-P-C`` flat; ``False`` yields ``C-P-P`` flat (HF / Gemma4 native). When ``flatten_patches=False``, ``True`` returns ``[N, Ph, Pw, C]`` and ``False`` returns ``[N, C, Ph, Pw]``. rr rrij)indexing)rz)dtype) shaperZnnr rviewpermutereshapemeshgridarangestackonesrX)rFrrrrrcrOrPr}r~pad_hpad_wnhnwpatchesy_idxx_idxcoordvalids r%patchify_imagers,iiGAq! FB B! q2v{a"f"a"f"hh!!%%cAua+?@))1a"Wa2gB((1b"b"-55aAq!D6E'//"b2gk27??[]_aceghKi((1b"b"-55aAq!D6E'//"a"frk27??[]_`bdfhKi~~ell2&6 R8HSWX u U]]2. b0ABJ 27%**5u$$ Nr'cPeZdZdZ ddeeeeeffdedeffd ZdZ xZ S)PatchifyzSTransform an image into patches with corresponding coordinates and type indicators.rrrctt|t|tr|n||f|_||_||_yr)r@rAr0rJrrr)rDrrrrEs r%rAzPatchify.__init__*s8 (2:u(E*JXbKc.*r'ct|tjrtjj |}t ||j |j|j\}}}|||dS)a! Args: img: A PIL Image or tensor of shape [C, H, W] Returns: A dictionary containing: - patches: Tensor of shape [N, P*P*C] if flatten_patches=True, or [N, Ph, Pw, C] if flatten_patches=False and channels_last=True or [N, C*P*P] / [N, C, Ph, Pw] if channels_last=False - patch_coord: Tensor of shape [N, 2] with (y, x) coordinates - patch_valid: Valid indicator (all 1s for non-padding patches) )rr)r patch_coord patch_valid) r0r r r to_tensorrrrr)rDrFrrrs r%rRzPatchify.forward5sj c5;; '''11#6C .  OO 00,, !     r')TT) rTrUrVrWrrrrXrArRr\r]s@r%rr'sF] %)"& +c5c?23 +" + + r'r)rrSTNgh㈵>)r)TTTT)(rWrr?rtypingrrrrrrrZPILr torchvisionr torchvision.transformsr rj!torchvision.transforms.functionalr rrrrrXrYr8rCrModuler<r_rrrrr[rrr)r'r%rs  ?? "2?KK 35#'%) ^S/^#uS#X./^^! ^ E? ^  ^ 5%S/ !"^B,J79KI9VW?uxx?DjFjFZb588??b@SB588??SBl |%((//|D! $" 5 \\5#s(O55 5  5  5 5<<u||U\\5<<GH HI5p* uxx* r'