^j ddlZddlZddlZddlmZddlmZdefdZdZddZ ddZ dd Z d Z dd ejd ed ededef dZ dd ejd ededefdZy)N)nn)_calculate_fan_in_and_fan_outreturnc|t|dk(Sttdr)tj}|duxr|jdk(Sy)zACheck if targeting meta device (explicit arg or context manager).Nmetaget_default_deviceF)strhasattrtorchrtype)devicedefault_devices b/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/layers/weight_init.pyis_meta_devicersN 6{f$$u*+113T)Kn.A.AV.KK cd}||d|zz ks ||d|zzkDrtjdd|||z |z }|||z |z }|jd|zdz d|zdz |j|j |t j dz|j||j|||S)Ncddtj|tjdz zdz S)N?@)matherfsqrt)xs rnorm_cdfz _trunc_normal_..norm_cdfs(TXXa$))B-/00B66rzjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelr)minmax) warningswarnuniform_erfinv_mul_rradd_clamp_)tensormeanstdabrlus r_trunc_normal_r.s7 q1s7{q1s7{ 2 E!" $ !d(c!"A!d(c!"A OOAEAIq1uqy) NN KKdiim#$ KK MMaQM Mrcttj5t|||||cdddS#1swYyxYw)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this impl is similar to the PyTorch trunc_normal_, the bounds [a, b] are applied while sampling the normal with mean/std applied, therefore a, b args should be adjusted to match the range of mean, std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) N)r no_gradr.r'r(r)r*r+s r trunc_normal_r26s2. 7fdCA6777s.7ctj5t|dd|||j|j |ddd|S#1swY|SxYw)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsequently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) rrN)r r0r.r$r%r1s rtrunc_normal_tf_r4QsR. $vq#q!, Cd#$ M$ Ms 0AAc,t|\}}|dk(r|}n|dk(r|}n |dk(r||zdz }|z }|dk(r$t|tj|dz y|dk(rCt j 5|j tj|dddy|d k(rItjd |z}t j 5|j| |dddytd |#1swYyxYw#1swYyxYw) Nfan_infan_outfan_avgrtruncated_normalg۶%?r)normaluniformzinvalid distribution ) rr4rrr r0normal_r" ValueError) r'scalemode distributionr6r7denomvariancebounds rvariance_scaling_rFns3F;OFG x    '!Q&u}H))TYYx%8;M%MN  ! ]]_ 4 NNtyy2N 3 4 4  " !h,' ]]_ + OOUFE * + +0?@@ 4 4 + +s3&C>D >D Dc t|ddy)Nr6r9)rArB)rF)r's r lecun_normal_rHsf8:LMrmodulename init_bias head_biasclassifier_namect|tjtjtjtj fr|j |rTtjj|jtjj|j|ytjj|jdt|tjr8|j+tjj|j|yyyt|dr|jyy)N{Gz?r: init_weights) isinstancerLinearConv1dConv2dConv3d startswithinitzeros_weight constant_biasr2r rP)rIrJrKrLrMs rinit_weight_vitr\s&299biiBIIFG ??? + GGNN6== ) GG  fkk9 5 GG ! !&--T ! :&")),1H!!&++y92I,  ( )rc&t|tjr|j|rTtjj |j tjj|j|ytjj|j |jYd|vr+tjj|jdn(tjj |jyyt|tjrLt|j |j*tjj |jyyt|dr|jyy)Nmlpgư>r:rP)rQrrRrVrWrXrYrZr[xavier_uniform_r>rTrHr rP)rIrJrLrMs rinit_weight_jaxr`s &"))$ ??? + GGNN6== ) GG  fkk9 5 GG # #FMM 2{{&:?4- 6RWW^^\b\g\gMh' FBII &fmm$ ;; " GGNN6;; ' #  ( )r)rgr)rr6r;)rOrahead)rarb)r rr r torch.nn.initrboolrr.r2r4rFrHModuler floatr\r`rrrhs 7d F76:A2N %     ,%   r