^jrddlZddlZddlZddlmZddlZddlmcmZ ddlm Z ddl m Z ddl mZddlmZmZmZmZmZddlmZmZddlmZgd ZGd d ZGd d eZGddeZegZGddeZGddeZGddeZ GddeZ!dZ"GddeZ#GddeZ$GddeZ%Gdd eZ&Gd!d"eZ'Gd#d$eZ(Gd%d&eZ)Gd'd(eZ*Gd)d*eZ+Gd+d,eZ,Gd-d.eZ-Gd/d0eZ.Gd1d2eZ/y)3N)Sequence)Tensor) constraints) Distribution)_sum_rightmost broadcast_all lazy_propertytril_matrix_to_vecvec_to_tril_matrix)padsoftplus)_Number) AbsTransformAffineTransform CatTransformComposeTransformCorrCholeskyTransformCumulativeDistributionTransform ExpTransformIndependentTransformLowerCholeskyTransformPositiveDefiniteTransformPowerTransformReshapeTransformSigmoidTransformSoftplusTransform TanhTransformSoftmaxTransformStackTransformStickBreakingTransform Transformidentity_transformceZdZUdZdZej ed<ej ed<ddeddffd Z d Z e defd Z e dd Z e defd Zdd ZdZdZdZdZdZdZdZdZdZdZxZS)r!a Abstract class for invertible transformations with computable log det jacobians. They are primarily used in :class:`torch.distributions.TransformedDistribution`. Caching is useful for transforms whose inverses are either expensive or numerically unstable. Note that care must be taken with memoized values since the autograd graph may be reversed. For example while the following works with or without caching:: y = t(x) t.log_abs_det_jacobian(x, y).backward() # x will receive gradients. However the following will error when caching due to dependency reversal:: y = t(x) z = t.inv(y) grad(z.sum(), [y]) # error because z is x Derived classes should implement one or both of :meth:`_call` or :meth:`_inverse`. Derived classes that set `bijective=True` should also implement :meth:`log_abs_det_jacobian`. Args: cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. Attributes: domain (:class:`~torch.distributions.constraints.Constraint`): The constraint representing valid inputs to this transform. codomain (:class:`~torch.distributions.constraints.Constraint`): The constraint representing valid outputs to this transform which are inputs to the inverse transform. bijective (bool): Whether this transform is bijective. A transform ``t`` is bijective iff ``t.inv(t(x)) == x`` and ``t(t.inv(y)) == y`` for every ``x`` in the domain and ``y`` in the codomain. Transforms that are not bijective should at least maintain the weaker pseudoinverse properties ``t(t.inv(t(x)) == t(x)`` and ``t.inv(t(t.inv(y))) == t.inv(y)``. sign (int or Tensor): For bijective univariate transforms, this should be +1 or -1 depending on whether transform is monotone increasing or decreasing. Fdomaincodomain cache_sizereturnNcz||_d|_|dk(rn|dk(rd|_n tdt|y)Nr)NNzcache_size must be 0 or 1) _cache_size_inv _cached_x_y ValueErrorsuper__init__)selfr& __class__s i/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/distributions/transforms.pyr/zTransform.__init__`sB%&* ?  1_)D 89 9 cD|jj}d|d<|S)Nr+)__dict__copy)r0states r2 __getstate__zTransform.__getstate__ks" ""$f  r3c|jj|jjk(r|jjStd)Nz:Please use either .domain.event_dim or .codomain.event_dim)r$ event_dimr%r-r0s r2r:zTransform.event_dimps: ;; DMM$;$; ;;;(( (UVVr3cF|j}|t|}||_|S)z{ Returns the inverse :class:`Transform` of this transform. This should satisfy ``t.inv.inv is t``. )r+_InverseTransform)r0invs r2r>z Transform.invvs( ii ;#D)CDI r3ct)z Returns the sign of the determinant of the Jacobian, if applicable. In general this only makes sense for bijective transforms. NotImplementedErrorr;s r2signzTransform.signs "!r3c|j|k(r|St|jtjurt||St t|d)Nr&z.with_cache is not implemented)r*typer/r!rAr0r&s r2 with_cachezTransform.with_cachesU   z )K :  )"4"4 44:4 4!T$ZL0N"OPPr3c ||uSNr0others r2__eq__zTransform.__eq__s u}r3c&|j| SrI)rMrKs r2__ne__zTransform.__ne__s;;u%%%r3c|jdk(r|j|S|j\}}||ur|S|j|}||f|_|S)z2 Computes the transform `x => y`. r)r*_callr,)r0xx_oldy_oldys r2__call__zTransform.__call__sY   q ::a= '' u :L JJqMa4r3c|jdk(r|j|S|j\}}||ur|S|j|}||f|_|S)z1 Inverts the transform `y => x`. r)r*_inverser,)r0rUrSrTrRs r2 _inv_callzTransform._inv_calls[   q ==# #'' u :L MM! a4r3ct)zD Abstract method to compute forward transformation. r@r0rRs r2rQzTransform._call "!r3ct)zD Abstract method to compute inverse transformation. r@r0rUs r2rXzTransform._inverser\r3ct)zU Computes the log det jacobian `log |dy/dx|` given input and output. r@r0rRrUs r2log_abs_det_jacobianzTransform.log_abs_det_jacobianr\r3c4|jjdzS)Nz())r1__name__r;s r2__repr__zTransform.__repr__s~~&&--r3c|S)z{ Infers the shape of the forward computation, given the input shape. Defaults to preserving shape. rJr0shapes r2 forward_shapezTransform.forward_shape  r3c|S)z} Infers the shapes of the inverse computation, given the output shape. Defaults to preserving shape. rJrfs r2 inverse_shapezTransform.inverse_shaperir3r)r'r!r))rc __module__ __qualname____doc__ bijectiver Constraint__annotations__intr/r8propertyr:r>rBrGrMrOrVrYrQrXrardrhrk __classcell__r1s@r2r!r!/s*XI  " ""$$$ 3 t  W3WW   "c""Q&  " " " .r3r!ceZdZdZdeddffd ZejddZejdd Z e de fd Z e de fd Ze defd Zdd ZdZdZdZdZdZdZxZS)r=z| Inverts a single :class:`Transform`. This class is private; please instead use the ``Transform.inv`` property. transformr'NcHt||j||_yNrD)r.r/r*r+)r0ryr1s r2r/z_InverseTransform.__init__s  I$9$9:&/ r3F is_discretec\|j td|jjSN_inv must not be None)r+AssertionErrorr%r;s r2r$z_InverseTransform.domains* 99  !89 9yy!!!r3c\|j td|jjSr)r+rr$r;s r2r%z_InverseTransform.codomains* 99  !89 9yyr3c\|j td|jjSr)r+rrqr;s r2rqz_InverseTransform.bijectives( 99  !89 9yy"""r3c\|j td|jjSr)r+rrBr;s r2rBz_InverseTransform.signs& 99  !89 9yy~~r3c|jSrI)r+r;s r2r>z_InverseTransform.invs yyr3cz|j td|jj|jSr)r+rr>rGrFs r2rGz_InverseTransform.with_caches3 99  !89 9xx"":.222r3ct|tsy|j td|j|jk(S)NFr) isinstancer=r+rrKs r2rMz_InverseTransform.__eq__s9%!23 99  !89 9yyEJJ&&r3c`|jjdt|jdS)N())r1rcreprr+r;s r2rdz_InverseTransform.__repr__ s)..))*!DO+>r3cf|j td|jj|Sr)r+rrYr[s r2rVz_InverseTransform.__call__ s- 99  !89 9yy""1%%r3cj|j td|jj|| Sr)r+rrar`s r2raz&_InverseTransform.log_abs_det_jacobians2 99  !89 9 ..q!444r3c8|jj|SrI)r+rkrfs r2rhz_InverseTransform.forward_shapeyy&&u--r3c8|jj|SrI)r+rhrfs r2rkz_InverseTransform.inverse_shaperr3rm)rcrnrorpr!r/rdependent_propertyr$r%ruboolrqrtrBr>rGrMrdrVrarhrkrvrws@r2r=r=s 0)00$[##6"7" $[##6 7 #4## c Y3 '?& 5 ..r3r=c eZdZdZddeededdffd ZdZe jd d Z e jd d Z e defd Ze defd ZedefdZddZdZdZdZdZdZxZS)rab Composes multiple transforms in a chain. The transforms being composed are responsible for caching. Args: parts (list of :class:`Transform`): A list of transforms to compose. cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. partsr&r'Nc~|r|Dcgc]}|j|}}t| |||_ycc}wr{)rGr.r/r)r0rr&partr1s r2r/zComposeTransform.__init__)s? =BCTT__Z0CEC J/ Ds:cVt|tsy|j|jk(SNF)rrrrKs r2rMzComposeTransform.__eq__/s#%!12zzU[[((r3Fr|cB|jstjS|jdj}|jdjj }t |jD]R}||jj |jj z z }t||jj }T||j krtd|d|j ||j kDr#tj|||j z }|S)Nr event_dim z must be >= domain.event_dim ) rrrealr$r%r:reversedmaxr independent)r0r$r:rs r2r$zComposeTransform.domain4szz## #A%%JJrN++55 TZZ( >D ..1H1HH HIIt{{'<'<=I > v'' ' YK'DVEUEUDVW  v'' ' ,,VYAQAQ5QRF r3c0|jstjS|jdj}|jdjj }|jD]R}||jj |jj z z }t ||jj }T||j krtd|d|j ||j kDr#tj|||j z }|S)Nrrrz must be >= codomain.event_dim ) rrrr%r$r:rrr)r0r%r:rs r2r%zComposeTransform.codomainGszz## #::b>**JJqM((22 JJ @D 004;;3H3HH HIIt}}'>'>?I @ x)) ) YK'FxGYGYFZ[  x)) )"..xXEWEW9WXHr3c:td|jDS)Nc34K|]}|jywrIrq).0ps r2 z-ComposeTransform.bijective..\s311;;3)allrr;s r2rqzComposeTransform.bijectiveZs3 333r3cJd}|jD]}||jz}|SNr))rrB)r0rBrs r2rBzComposeTransform.sign^s, !A!&&=D ! r3c|j}|Dtt|jDcgc]}|jc}}||_||_|Scc}wrI)r+rrrr>)r0r>rs r2r>zComposeTransform.invesKii ;"8DJJ3G#HaAEE#HICDICH $IsAcR|j|k(r|St|j|Sr{)r*rrrFs r2rGzComposeTransform.with_cachens&   z )K zBBr3c8|jD] }||} |SrI)r)r0rRrs r2rVzComposeTransform.__call__ss#JJ DQA r3c p|jstj|S|g}|jddD]}|j||d|j|g}|jj }t |j|dd|ddD]x\}}}|jt|j||||jj z ||jj |jj z z }ztjtj|S)Nrr))rtorch zeros_likeappendr$r:ziprrar% functoolsreduceoperatoradd)r0rRrUxsrtermsr:s r2raz%ComposeTransform.log_abs_det_jacobianxszz##A& &SJJsO $D IId2b6l # $ ! KK)) djj"Sb'2ab6: IJD!Q LL--a3YAVAV5V  004;;3H3HH HI  I e44r3cJ|jD]}|j|}|SrI)rrhr0rgrs r2rhzComposeTransform.forward_shapes*JJ .D&&u-E . r3c\t|jD]}|j|}|SrI)rrrkrs r2rkzComposeTransform.inverse_shapes/TZZ( .D&&u-E . r3c|jjdz}|dj|jDcgc]}|j c}z }|dz }|Scc}w)Nz( z, z ))r1rcjoinrrd)r0 fmt_stringrs r2rdzComposeTransform.__repr__sT^^,,y8 innDJJ%Gqajjl%GHH e &HsA rlrm)rcrnrorplistr!rtr/rMrrr$r%r rrqrBrur>rGrVrarhrkrdrvrws@r2rrsd9o3t ) $[##67"$[##67"4444c YC  5*  r3rc eZdZdZ ddedededdffd ZddZejd d Z ejd d Z e de fd Ze defdZdZdZdZdZdZdZxZS)ra Wrapper around another transform to treat ``reinterpreted_batch_ndims``-many extra of the right most dimensions as dependent. This has no effect on the forward or backward transforms, but does sum out ``reinterpreted_batch_ndims``-many of the rightmost dimensions in :meth:`log_abs_det_jacobian`. Args: base_transform (:class:`Transform`): A base transform. reinterpreted_batch_ndims (int): The number of extra rightmost dimensions to treat as dependent. base_transformreinterpreted_batch_ndimsr&r'Nc`t|||j||_||_yr{)r.r/rGrr)r0rrr&r1s r2r/zIndependentTransform.__init__s0 J/,77 C)B&r3ch|j|k(r|St|j|j|Sr{)r*rrrrFs r2rGzIndependentTransform.with_caches5   z )K#   !?!?J  r3Fr|cjtj|jj|jSrI)rrrr$rr;s r2r$zIndependentTransform.domains.&&    & &(F(F  r3cjtj|jj|jSrI)rrrr%rr;s r2r%zIndependentTransform.codomains.&&    ( ($*H*H  r3c.|jjSrI)rrqr;s r2rqzIndependentTransform.bijectives"",,,r3c.|jjSrI)rrBr;s r2rBzIndependentTransform.signs""'''r3c|j|jjkr td|j |SNToo few dimensions on input)dimr$r:r-rr[s r2rQzIndependentTransform._calls7 557T[[** *:; ;""1%%r3c|j|jjkr td|jj |Sr)rr%r:r-rr>r^s r2rXzIndependentTransform._inverses= 557T]],, ,:; ;""&&q))r3cj|jj||}t||j}|SrI)rrarr)r0rRrUresults r2raz)IndependentTransform.log_abs_det_jacobians1$$99!Q?(F(FG r3cz|jjdt|jd|jdS)Nrz, r)r1rcrrrr;s r2rdzIndependentTransform.__repr__s:..))*!D1D1D,E+FbIgIgHhhijjr3c8|jj|SrI)rrhrfs r2rhz"IndependentTransform.forward_shape""0077r3c8|jj|SrI)rrkrfs r2rkz"IndependentTransform.inverse_shaperr3rlrm)rcrnrorpr!rtr/rGrrr$r%rurrqrBrQrXrardrhrkrvrws@r2rrs " C!C$'C C  C $[##6 7 $[##6 7 -4--(c((& *  k88r3rc eZdZdZdZ ddej dej deddffd Ze jd Z e jd Z dd Z d Zd ZdZdZdZxZS)ra Unit Jacobian transform to reshape the rightmost part of a tensor. Note that ``in_shape`` and ``out_shape`` must have the same number of elements, just as for :meth:`torch.Tensor.reshape`. Arguments: in_shape (torch.Size): The input event shape. out_shape (torch.Size): The output event shape. cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. (Default 0.) Tin_shape out_shaper&r'Nctj||_tj||_|jj |jj k7r t dt ||y)Nz6in_shape, out_shape have different numbers of elementsrD)rSizerrnumelr-r.r/)r0rrr&r1s r2r/zReshapeTransform.__init__sc  8, I. ==   DNN$8$8$: :UV V J/r3cptjtjt|jSrI)rrrlenrr;s r2r$zReshapeTransform.domain s&&&{'7'7T]]9KLLr3cptjtjt|jSrI)rrrrrr;s r2r%zReshapeTransform.codomains&&&{'7'7T^^9LMMr3ch|j|k(r|St|j|j|Sr{)r*rrrrFs r2rGzReshapeTransform.with_caches,   z )K t~~*UUr3c|jd|jt|jz }|j ||j zSrI)rgrrrreshaper)r0rR batch_shapes r2rQzReshapeTransform._calls?gg<#dmm*< <= yyt~~566r3c|jd|jt|jz }|j ||j zSrI)rgrrrrr)r0rUrs r2rXzReshapeTransform._inverses?gg=#dnn*= => yyt}}455r3c|jd|jt|jz }|j |SrI)rgrrr new_zeros)r0rRrUrs r2raz%ReshapeTransform.log_abs_det_jacobian"s6gg<#dmm*< <= {{;''r3c t|t|jkr tdt|t|jz }||d|jk7rtd||dd|j|d||jzSNrzShape mismatch: expected z but got )rrr-rr0rgcuts r2rhzReshapeTransform.forward_shape&s u:DMM* *:; ;%j3t}}-- ;$-- '+E#$K= $--Q Tc{T^^++r3c t|t|jkr tdt|t|jz }||d|jk7rtd||dd|j|d||jzSr)rrr-rrs r2rkzReshapeTransform.inverse_shape0s u:DNN+ +:; ;%j3t~~.. ;$.. (+E#$K= $..AQR Tc{T]]**r3rlrm)rcrnrorprqrrrtr/rrr$r%rGrQrXrarhrkrvrws@r2rrs I  0** 0:: 0 0  0##M$M##N$NV 76(,+r3rc`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rz8 Transform via the mapping :math:`y = \exp(x)`. Tr)c"t|tSrI)rrrKs r2rMzExpTransform.__eq__E%..r3c"|jSrI)expr[s r2rQzExpTransform._callH uuwr3c"|jSrIlogr^s r2rXzExpTransform._inverseKrr3c|SrIrJr`s r2raz!ExpTransform.log_abs_det_jacobianNr3Nrcrnrorprrr$positiver%rqrBrMrQrXrarJr3r2rr;s=  F##HI D/r3rceZdZdZej Zej ZdZdde de ddffd Z ddZ e de fd Zd Zd Zd Zd ZdZdZxZS)rzD Transform via the mapping :math:`y = x^{\text{exponent}}`. Texponentr&r'NcJt||t|\|_yr{)r.r/rr)r0rr&r1s r2r/zPowerTransform.__init__[s" J/(2r3cR|j|k(r|St|j|Sr{)r*rrrFs r2rGzPowerTransform.with_cache_s&   z )Kdmm CCr3c6|jjSrI)rrBr;s r2rBzPowerTransform.signds}}!!##r3ct|tsy|jj|jj j Sr)rrreqritemrKs r2rMzPowerTransform.__eq__hs:%0}}/335::<|jd|jz Srrr^s r2rXzPowerTransform._inversepsuuQ&''r3c^|j|z|z jjSrI)rabsrr`s r2raz#PowerTransform.log_abs_det_jacobianss( !A%**,0022r3cXtj|t|jddSNrgrJrbroadcast_shapesgetattrrrfs r2rhzPowerTransform.forward_shapev"%%eWT]]GR-PQQr3cXtj|t|jddSrrrfs r2rkzPowerTransform.inverse_shapeyr r3rlrm)rcrnrorprrr$r%rqrrtr/rGr rBrMrQrXrarhrkrvrws@r2rrRs ! !F##HI33S33D $c$$= $(3RRr3rctj|j}tjtj||j d|j z SN?minr)rfinfodtypeclampsigmoidtinyeps)rRrs r2_clipped_sigmoidr}s< KK E ;;u}}Q'UZZS599_ MMr3c`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rzg Transform via the mapping :math:`y = \frac{1}{1 + \exp(-x)}` and :math:`x = \text{logit}(y)`. Tr)c"t|tSrI)rrrKs r2rMzSigmoidTransform.__eq__%!122r3ct|SrI)rr[s r2rQzSigmoidTransform._calls ""r3ctj|j}|j|jd|j z }|j | jz Sr )rrrrrrrlog1p)r0rUrs r2rXzSigmoidTransform._inversesK AGG$ GG eiiG 8uuw1"%%r3c\tj|  tj|z SrI)Fr r`s r2raz%SigmoidTransform.log_abs_det_jacobians! A2A..r3N)rcrnrorprrr$ unit_intervalr%rqrBrMrQrXrarJr3r2rrs=  F((HI D3#& /r3rc`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rz Transform via the mapping :math:`\text{Softplus}(x) = \log(1 + \exp(x))`. The implementation reverts to the linear function when :math:`x > 20`. Tr)c"t|tSrI)rrrKs r2rMzSoftplusTransform.__eq__s%!233r3ct|SrIr r[s r2rQzSoftplusTransform._calls {r3cb| jjj|zSrI)expm1negrr^s r2rXzSoftplusTransform._inverses'zz|!%%'!++r3ct|  SrIr$r`s r2raz&SoftplusTransform.log_abs_det_jacobians! }r3NrrJr3r2rrs=   F##HI D4,r3rcneZdZdZej ZejddZdZ dZ dZ dZ dZ d Zy ) ra Transform via the mapping :math:`y = \tanh(x)`. It is equivalent to .. code-block:: python ComposeTransform( [ AffineTransform(0.0, 2.0), SigmoidTransform(), AffineTransform(-1.0, 2.0), ] ) However this might not be numerically stable, thus it is recommended to use `TanhTransform` instead. Note that one should use `cache_size=1` when it comes to `NaN/Inf` values. grTr)c"t|tSrI)rrrKs r2rMzTanhTransform.__eq__s%//r3c"|jSrI)tanhr[s r2rQzTanhTransform._calls vvxr3c,tj|SrI)ratanhr^s r2rXzTanhTransform._inverses{{1~r3cVdtjd|z td|zz zS)N@g)mathrr r`s r2raz"TanhTransform.log_abs_det_jacobians*dhhsma'(4!8*<<==r3N)rcrnrorprrr$intervalr%rqrBrMrQrXrarJr3r2rrsF,  F#{##D#.HI D0 >r3rcReZdZdZej ZejZdZ dZ dZ y)rz*Transform via the mapping :math:`y = |x|`.c"t|tSrI)rrrKs r2rMzAbsTransform.__eq__rr3c"|jSrI)rr[s r2rQzAbsTransform._callrr3c|SrIrJr^s r2rXzAbsTransform._inverserr3N) rcrnrorprrr$rr%rMrQrXrJr3r2rrs*5   F##H/r3rc eZdZdZdZ ddeezdeezdededdf fd Ze defd Z e jd d Z e jd dZddZdZe deezfdZdZdZdZdZdZxZS)ra Transform via the pointwise affine mapping :math:`y = \text{loc} + \text{scale} \times x`. Args: loc (Tensor or float): Location parameter. scale (Tensor or float): Scale parameter. event_dim (int): Optional size of `event_shape`. This should be zero for univariate random variables, 1 for distributions over vectors, 2 for distributions over matrices, etc. Tlocscaler:r&r'NcPt||||_||_||_yr{)r.r/r8r9 _event_dim)r0r8r9r:r&r1s r2r/zAffineTransform.__init__s* J/ #r3c|jSrI)r;r;s r2r:zAffineTransform.event_dim s r3Fr|c|jdk(rtjStjtj|jSNrr:rrrr;s r2r$zAffineTransform.domain9 >>Q ## #&&{'7'7HHr3c|jdk(rtjStjtj|jSr>r?r;s r2r%zAffineTransform.codomainr@r3c~|j|k(r|St|j|j|j|Sr{)r*rr8r9r:rFs r2rGzAffineTransform.with_caches7   z )K HHdjj$..Z  r3c8t|tsyt|jtr4t|jtr|j|jk7r7y|j|jk(j j syt|j tr5t|j tr|j |j k7ryy|j |j k(j j syy)NFT)rrr8rrrr9rKs r2rMzAffineTransform.__eq__#s%1 dhh (Z 7-Kxx599$HH )..0557 djj' *z%++w/OzzU[[( JJ%++-22499;r3ct|jtr6t|jdkDrdSt|jdkrdSdS|jj S)Nrr)r)rr9rfloatrBr;s r2rBzAffineTransform.sign7sR djj' *djj)A-1 Utzz9JQ9N2 UTU Uzz  r3c:|j|j|zzSrIr8r9r[s r2rQzAffineTransform._call=sxx$**q.((r3c:||jz |jz SrIrGr^s r2rXzAffineTransform._inverse@sDHH  **r3c|j}|j}t|tr3t j |t jt|}n#t j|j}|jrQ|jd|j dz}|j|jd}|d|j }|j|S)N)rr)rgr9rrr full_liker1rrr:sizeviewsumexpand)r0rRrUrgr9r result_sizes r2raz$AffineTransform.log_abs_det_jacobianCs  eW %__QU(<=FYYu%))+F >> ++-(94>>/:UBK[[-11"5F+T^^O,E}}U##r3c tj|t|jddt|jddSrrrr r8r9rfs r2rhzAffineTransform.forward_shapeP7%% 7488Wb174::wPR3S  r3c tj|t|jddt|jddSrrQrfs r2rkzAffineTransform.inverse_shapeUrRr3rrrm)rcrnrorprqrrErtr/rur:rrr$r%rGrMrBrQrXrarhrkrvrws@r2rrs I  $ e^ $~ $ $  $  $3$[##6I7I $[##6I7I  (!fsl!! )+ $  r3rcdeZdZdZej ZejZdZ dZ dZ d dZ dZ dZy) ra Transforms an unconstrained real vector :math:`x` with length :math:`D*(D-1)/2` into the Cholesky factor of a D-dimension correlation matrix. This Cholesky factor is a lower triangular matrix with positive diagonals and unit Euclidean norm for each row. The transform is processed as follows: 1. First we convert x into a lower triangular matrix in row order. 2. For each row :math:`X_i` of the lower triangular part, we apply a *signed* version of class :class:`StickBreakingTransform` to transform :math:`X_i` into a unit Euclidean length vector using the following steps: - Scales into the interval :math:`(-1, 1)` domain: :math:`r_i = \tanh(X_i)`. - Transforms into an unsigned domain: :math:`z_i = r_i^2`. - Applies :math:`s_i = StickBreakingTransform(z_i)`. - Transforms back into signed domain: :math:`y_i = sign(r_i) * \sqrt{s_i}`. Tctj|}tj|jj}|j d|zd|z }t |d}|dz}d|z jjd}|tj|jd|j|jz}|t|dddfddgd z}|S) Nrr)rdiag)rdevice.rvalue) rr,rrrrr sqrtcumprodeyergrZr )r0rRrrzz1m_cumprod_sqrtrUs r2rQzCorrCholeskyTransform._callps JJqMkk!''"&& GGSa#gG . qr * qDE<<>11"5  !''"+QWWQXXF F $S#2#X.Aa@ @r3c,dtj||zdz }t|dddfddgd}t|d}t|d}||j z }|j |j j z dz }|S) Nr)rr.rr[rWrY)rcumsumr r r]rr')r0rUy_cumsumy_cumsum_shiftedy_vec y_cumsum_vectrRs r2rXzCorrCholeskyTransform._inversesu||AEr22xSbS1Aq6C"12.)*:D \'') ) WWY (A -r3Ncd||zjdz }t|d}d|jjdz}d|t d|zzt jdz jdz}||zS)Nr)rrdrW?r0)rer rrMr r1)r0rRrU intermediates y1m_cumsumy1m_cumsum_trilstick_breaking_logdet tanh_logdets r2raz*CorrCholeskyTransform.log_abs_det_jacobians !a%B// -ZbA #&;&;&=&A&A"&E EAa 00488C=@EE"EMM ${22r3ct|dkr td|d}tdd|zzdzdz}||dz zdz|k7r td|dd||fzS)Nr)rrg?rYrmz.Input is not a flattened lower-diagonal number)rr-round)r0rgNDs r2rhz#CorrCholeskyTransform.forward_shapest u:>:; ; "I 4!a%:; ; 9b !23 3 "I QK1 SbzQD  r3rI)rcrnrorpr real_vectorr$ corr_choleskyr%rqrQrXrarhrkrJr3r2rr[s=  $ $F((HI   3#!r3rc^eZdZdZej ZejZdZ dZ dZ dZ dZ y)ra< Transform from unconstrained space to the simplex via :math:`y = \exp(x)` then normalizing. This is not bijective and cannot be used for HMC. However this acts mostly coordinate-wise (except for the final normalization), and thus is appropriate for coordinate-wise optimization algorithms. c"t|tSrI)rrrKs r2rMzSoftmaxTransform.__eq__rr3c||}||jdddz j}||jddz S)NrTr)rrrM)r0rRlogprobsprobss r2rQzSoftmaxTransform._calls@HLLT2155::<uyyT***r3c&|}|jSrIr)r0rUrs r2rXzSoftmaxTransform._inversesyy{r3c8t|dkr td|SNr)rrxrfs r2rhzSoftmaxTransform.forward_shape u:>:; ; r3c8t|dkr td|Srrxrfs r2rkzSoftmaxTransform.inverse_shaperr3N)rcrnrorprryr$simplexr%rMrQrXrhrkrJr3r2rrs8 $ $F""H3+  r3rcheZdZdZej ZejZdZ dZ dZ dZ dZ dZdZy ) r a Transform from unconstrained space to the simplex of one additional dimension via a stick-breaking process. This transform arises as an iterated sigmoid transform in a stick-breaking construction of the `Dirichlet` distribution: the first logit is transformed via sigmoid to the first probability and the probability of everything else, and then the process recurses. This is bijective and appropriate for use in HMC; however it mixes coordinates together and is less appropriate for optimization. Tc"t|tSrI)rr rKs r2rMzStickBreakingTransform.__eq__%!788r3c(|jddz|j|jdjdz }t||j z }d|z j d}t |ddgdt |ddgdz}|S)Nrr)rr[)rgnew_onesrerrr^r )r0rRoffsetra z_cumprodrUs r2rQzStickBreakingTransform._callsq1::aggbk#:#A#A"#EE Q- .UOOB' Aq6 #c)aV1&E Er3c|dddf}|jd|j|jdjdz }d|jdz }tj|tj |j j}|j|jz |jz}|S)N.rr))r) rgrrerrrrrr)r0rUy_croprsfrRs r2rXzStickBreakingTransform._inverses38qzz&,,r*:;BB2FF r" "[[QWW!5!:!: ; JJL2668 #fjjl 2r3c,|jddz|j|jdjdz }||jz }| t j |z|dddfjzj d}|S)Nrr).)rgrrerr logsigmoidrM)r0rRrUrdetJs r2raz+StickBreakingTransform.log_abs_det_jacobiansq1::aggbk#:#A#A"#EE  Q\\!_$qcrc{'88==bA r3cRt|dkr td|dd|ddzfzSNr)rrrxrfs r2rhz$StickBreakingTransform.forward_shape5 u:>:; ;SbzU2Y],,,r3cRt|dkr td|dd|ddz fzSrrxrfs r2rkz$StickBreakingTransform.inverse_shaperr3N)rcrnrorprryr$rr%rqrMrQrXrarhrkrJr3r2r r sB  $ $F""HI9- -r3r cteZdZdZej ej dZejZ dZ dZ dZ y)rz Transform from unconstrained matrices to lower-triangular matrices with nonnegative diagonal entries. This is useful for parameterizing positive definite matrices in terms of their Cholesky factorization. rYc"t|tSrI)rrrKs r2rMzLowerCholeskyTransform.__eq__rr3c|jd|jddjjzSNrrl)dim1dim2)trildiagonalr diag_embedr[s r2rQzLowerCholeskyTransform._call4vvbzAJJBRJ8<<>IIKKKr3c|jd|jddjjzSr)rrrrr^s r2rXzLowerCholeskyTransform._inverserr3N) rcrnrorprrrr$lower_choleskyr%rMrQrXrJr3r2rr s?%[ $ $[%5%5q 9F))H9LLr3rcteZdZdZej ej dZejZ dZ dZ dZ y)rzN Transform from unconstrained matrices to positive-definite matrices. rYc"t|tSrI)rrrKs r2rMz PositiveDefiniteTransform.__eq__(s%!:;;r3c@t|}||jzSrI)rmTr[s r2rQzPositiveDefiniteTransform._call+s $ " $Q '144xr3crtjj|}tj |SrI)rlinalgcholeskyrr>r^s r2rXz"PositiveDefiniteTransform._inverse/s* LL ! !! $%'++A..r3N) rcrnrorprrrr$positive_definiter%rMrQrXrJr3r2rr s=%[ $ $[%5%5q 9F,,H</r3rc eZdZUdZeeed< ddeededeedzdeddf fd Z e defd Z e defd Z dd Z d ZdZdZedefdZej*dZej*dZxZS)ra Transform functor that applies a sequence of transforms `tseq` component-wise to each submatrix at `dim`, of length `lengths[dim]`, in a way compatible with :func:`torch.cat`. Example:: x0 = torch.cat([torch.range(1, 10), torch.range(1, 10)], dim=0) x = torch.cat([x0, x0], dim=0) t0 = CatTransform([ExpTransform(), identity_transform], dim=0, lengths=[10, 10]) t = CatTransform([t0, t0], dim=0, lengths=[20, 20]) y = t(x) transformsNtseqrlengthsr&r'ctd|Ds td|r|Dcgc]}|j|}}t||t ||_|dgt|j z}t ||_t|jt|j k7r8tdt|jdt|j d||_ ycc}w)Nc3<K|]}t|tywrIrr!rrjs r2rz(CatTransform.__init__..L::a+:0All elements of tseq must be Transform instancesrDr)z lengths (z) must match transforms (r) rrrGr.r/rrrrr)r0rrrr&rjr1s r2r/zCatTransform.__init__Es:T:: !ST T 6:;ALL,;D; J/t* ?cC00GG} t|| DOO 4 4 C -..GDOOH\G]]^_ .]811;;8r)rrr;s r2r:zCatTransform.event_dim[8888r3c,t|jSrI)rMrr;s r2lengthzCatTransform.length_s4<<  r3c||j|k(r|St|j|j|j|SrI)r*rrrrrFs r2rGzCatTransform.with_cachecs2   z )KDOOTXXt||ZPPr3c|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|jg}d}t |j |j D]>\}}|j|j||}|j||||z}@tj||jS) Ndim  out of range for tensor with dimensionsx.size() =  must equal length rrd) rrrKrrrrnarrowrrcat)r0rRyslicesstarttransrxslices r2rQzCatTransform._callhsDHH.quuw. txxj >quuwi{S  66$(( t{{ * $((4txx(8'99LT[[MZ  $,,? #ME6XXdhhv6F NN5= )FNE #yydhh//r3c|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|jg}d}t |j |j D]G\}}|j|j||}|j|j|||z}Itj||jS) Nrrry.size(rrrrd) rrrKrrrrrrr>rr)r0rUxslicesrrryslices r2rXzCatTransform._inverseysDHH.quuw. txxj >quuwi{S  66$(( t{{ * $((4txx(8'99LT[[MZ  $,,? #ME6XXdhhv6F NN599V, -FNE #yydhh//r3cf|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|j|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|jg}d }t |j |j D]\}}|j|j||}|j|j||}|j||} |j|jkr#t| |j|jz } |j| ||z}|j} | d k\r| |jz } | |jz} | d krtj|| St|S) Nr out of range for x with rrrr out of range for y with rrrd)rrrKrrrrrrar:rrrrrM) r0rRrU logdetjacsrrrrr logdetjacrs r2raz!CatTransform.log_abs_det_jacobians=DHH.quuw. txxj 9!%%'+N  66$(( t{{ * $((4txx(8'99LT[[MZ DHH.quuw. txxj 9!%%'+N  66$(( t{{ * $((4txx(8'99LT[[MZ   $,,? #ME6XXdhhv6FXXdhhv6F2266BI/*9dnnu6VW   i (FNE #hh !8-CDNN" 799ZS1 1z? "r3c:td|jDS)Nc34K|]}|jywrIrrs r2rz)CatTransform.bijective..rrrrr;s r2rqzCatTransform.bijectiverr3ctj|jDcgc]}|jc}|j|j Scc}wrI)rrrr$rrr0rjs r2r$zCatTransform.domains:# /!QXX /4<<  /Actj|jDcgc]}|jc}|j|j Scc}wrI)rrrr%rrrs r2r%zCatTransform.codomains:!% 1AQZZ 1488T\\  1r)rNrrm)rcrnrorprr!rsrrtr/r r:rrGrQrXrarurrqrrr$r%rvrws@r2rr4s Y (, y!#%     ,9399!!!Q 0"0"##J9499## $ ## $ r3rc eZdZUdZeeed< ddeedededdffd Z dd Z d Z d Z d Z d ZedefdZej&dZej&dZxZS)raW Transform functor that applies a sequence of transforms `tseq` component-wise to each submatrix at `dim` in a way compatible with :func:`torch.stack`. Example:: x = torch.stack([torch.range(1, 10), torch.range(1, 10)], dim=1) t = StackTransform([ExpTransform(), identity_transform], dim=1) y = t(x) rrrr&r'Nctd|Ds td|r|Dcgc]}|j|}}t||t ||_||_ycc}w)Nc3<K|]}t|tywrIrrs r2rz*StackTransform.__init__..rrrrD)rrrGr.r/rrr)r0rrr&rjr1s r2r/zStackTransform.__init__se:T:: !ST T 6:;ALL,;D; J/t*quuwi{S  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl  QA *MFE NN5= ) *{{711r3c |j |jcxkr|jks,ntd|jd|jd|j|jt|jk7rJtd|jd|j|jdt|jg}t |j ||jD]%\}}|j|j|'tj||jS)Nrrrrrrrd) rrrKrrrrrr>rr)r0rUrrrs r2rXzStackTransform._inversesDHH.quuw. txxj >quuwi{S  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl  QA .MFE NN599V, - .{{711r3c r|j |jcxkr|jks,ntd|jd|jd|j|jt|jk7rJtd|jd|j|jdt|j|j |jcxkr|jks,ntd|jd|jd|j|jt|jk7rJtd|jd|j|jdt|jg}|j |}|j |}t |||jD]'\}}}|j|j||)tj||j S) Nrrrrrrrrrd) rrrKrrrrrrarr) r0rRrUrrrrrrs r2raz#StackTransform.log_abs_det_jacobiansDHH.quuw. txxj 9!%%'+N  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl DHH.quuw. txxj 9!%%'+N  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl  ++a.++a.%('4??%K J !FFE   e88H I J{{:48844r3c:td|jDS)Nc34K|]}|jywrIrrs r2rz+StackTransform.bijective..rrrr;s r2rqzStackTransform.bijectiverr3ctj|jDcgc]}|jc}|jScc}wrI)rrrr$rrs r2r$zStackTransform.domains1  DOO!Dq!((!DdhhOO!DActj|jDcgc]}|jc}|jScc}wrI)rrrr%rrs r2r%zStackTransform.codomain!s1  doo!F!**!FQQ!FrrTrm)rcrnrorprr!rsrrtr/rGrrQrXrarurrqrrr$r%rvrws@r2rrs YJK Y' .1 CF   E H 2 2509499##P$P##R$Rr3rceZdZdZdZej ZdZdde de ddffd Z e dejdzfd Zd Zd Zd Zdd ZxZS)raA Transform via the cumulative distribution function of a probability distribution. Args: distribution (Distribution): Distribution whose cumulative distribution function to use for the transformation. Example:: # Construct a Gaussian copula from a multivariate normal. base_dist = MultivariateNormal( loc=torch.zeros(2), scale_tril=LKJCholesky(2).sample(), ) transform = CumulativeDistributionTransform(Normal(0, 1)) copula = TransformedDistribution(base_dist, [transform]) Tr) distributionr&r'Nc4t||||_yr{)r.r/r)r0rr&r1s r2r/z(CumulativeDistributionTransform.__init__>s J/(r3c.|jjSrI)rsupportr;s r2r$z&CumulativeDistributionTransform.domainBs  (((r3c8|jj|SrI)rcdfr[s r2rQz%CumulativeDistributionTransform._callFs  $$Q''r3c8|jj|SrI)ricdfr^s r2rXz(CumulativeDistributionTransform._inverseIs  %%a((r3c8|jj|SrI)rlog_probr`s r2raz4CumulativeDistributionTransform.log_abs_det_jacobianLs  ))!,,r3cR|j|k(r|St|j|Sr{)r*rrrFs r2rGz*CumulativeDistributionTransform.with_cacheOs(   z )K.t/@/@ZXXr3rlrm)rcrnrorprqrr r%rBrrtr/rurrr$rQrXrarGrvrws@r2rr'st$I((H D)\)s)4)) ..5))()-Yr3r)0rr1rcollections.abcrrtorch.nn.functionalnn functionalrrtorch.distributionsr torch.distributions.distributionrtorch.distributions.utilsrrr r r r r torch.typesr__all__r!r=rr"rrrrrrrrrrrrr rrrrrrJr3r2rsd $ +9. 0ddNE. E.P}y}@&b)K89K8\I+yI+X9.(RY(RVN /y/2 0*>I*>Z 9  h ih VP!IP!f!y!H5-Y5-pLYL,/ /(K 9K \bRYbRJ+Yi+Yr3