^j^UdZddlmZddlZddlZddlmZmZmZm Z ddl Z ddl m Z m Z erddlmZmZddlmZgdZd ZdZdad ed <ej0Zd#d ZGd de Zd$dZGddeZGddZGddZGddZ GddZ!ee ze"zdzZ#ded<e#ezZ$ded<e$eze ze!zZ%ded<e%e&e%ze'e(e%fzZ)ded<Gdd Z*Gd!d"Z+y)%zxSpec types for controlling what is dynamic in compiled/exported code. Currently only supports unbacked dynamic shapes. ) annotationsN)Anycast TYPE_CHECKING TypeAlias)SymBoolSymInt)IteratorSequenceShapeEnv) ShapeVarIntVarSTATIC TensorSpec ObjectSpecDictSpecSeqSpec ParamsSpec ShapesSpecLeafSpec LeafIntSpecIntermediateSpecz zShapeEnv | None_SPEC_SHAPE_ENVcttSt5ttcdddSddlm}Gdd|}|atcdddS#1swYyxYw)z7Lazily build the singleton spec ShapeEnv (thread-safe).Nrr cPeZdZUdZehdZded<dfd Zdfd ZxZ S) *_get_spec_shape_env.._SpecShapeEnvuSpecial ShapeEnv used only at spec-definition time. Attribute access is blocked by default via ``__getattribute__``; only private (``_``-prefix) names and the explicit allowlist in ``_ALLOWED_PUBLIC`` pass through. SymInt arithmetic paths read a handful of public fields transparently (via the ``@record_shapeenv_event`` decorator and FX-cache machinery), so those are allowlisted. Everything else — evaluation, guard recording, deferred asserts, etc. — is blocked. During ``ShapeEnv.__init__`` itself, access is unrestricted because the base class touches its own public fields while bootstrapping. > bound_sympy is_recording var_to_range fx_node_cacheshould_record_eventszfrozenset[str]_ALLOWED_PUBLICc~tj|ddt| tj|ddy)N _init_doneFT)object __setattr__super__init__)self __class__s m/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/fx/experimental/dynamic_spec.pyr)z3_get_spec_shape_env.._SpecShapeEnv.__init__ms2""4u= """4t<c|jdrt| |St| dst| |S|t|jvrt| |St d|d)N_r%z_SpecShapeEnv: 'zd' is not allowed at spec-definition time. Use IntVar / ShapeVar only inside TensorSpec / ShapesSpec.) startswithr(__getattribute__typer# TypeError)r*namer+s r,r1z;_get_spec_shape_env.._SpecShapeEnv.__getattribute__ss|??3' 73D99w/ = 73D994:555 73D99&tf-67r-)returnNone)r4strr5r) __name__ __module__ __qualname____doc__ frozensetr#__annotations__r)r1 __classcell__r+s@r, _SpecShapeEnvrMs/ "/8  /O^  =  r-r@)r_SPEC_SHAPE_ENV_LOCK%torch.fx.experimental.symbolic_shapesr )r r@s r,_get_spec_shape_envrCCs[" 9  &"99 C1 H1 f(/s999s AAAc~eZdZdZej Z ddddd d fdZd dZd dZ d dZ xZ S) raIndicates that a scalar integer argument is dynamic (no implicit range). Unbacked dynamic shapes will represent the underlying value in the compiled graph. IntVar is a ``SymInt`` subclass backed by ``_SpecShapeEnv``, so arithmetic and comparisons compose naturally:: A = IntVar("a") B = IntVar("b") A + 1 # SymInt expr=a#0 + 1 A * B + 1 # SymInt expr=a#0*b#1 + 1 A > 0 # SymBool Such derived ``SymInt`` values may be used directly as leaf specs in ``TensorSpec`` / ``ParamsSpec`` Repr always includes a per-instance uid so two IntVars with the same name (or two anonymous ones) are distinguishable in logs:: IntVar() -> "IntVar(anon#0)" IntVar("offset") -> "IntVar(offset#1)" Example:: IntVar("num_heads") IntVar("offset", min=-100, max=100) IntVar("size", min=1, max=2048, optimization_hint=512) Nminmaxoptimization_hintcddlm}m}ddlm}ddlm}||nd|_ttj|_ ||_ ||_ ||_tj |jd|j|_t%} ||j"| t&|} t( |U| |||n| ||n|| j,|j"<y)Nr)_NO_HINTSymNode)int_oo) ValueRangesanon#) shape_envpytypehint)torch.fx.experimental.sym_noderJrKtorch.utils._sympy.numbersrLtorch.utils._sympy.value_rangesrMr4nextr _uid_counter_uidrFrGrHsympySymbol sympy_symrCintr(r)r ) r*r4rFrGrHrJrKrLrMenvnoder+s r,r)zIntVar.__init__s E5? ,D& ,,- !2 1TYYK&@A!# NN   ,7?C?C, (r-c|jd|jg}|j|jd|j|j|jd|j|j |jd|j t |jddj|dS)NrOzmin=zmax=zoptimization_hint=(z, )) r4rXrFappendrGrHr2r8join)r*partss r,__repr__zIntVar.__repr__sII;a {+, 88  LL4z* + 88  LL4z* +  ! ! - LL-d.D.D-EF Gt*%%&a %(8'9;;r-ct|SN)idr*s r,__hash__zIntVar.__hash__s $xr-ct|j|j|j|j|j dS)N)r2r4rFrGrH)r2r8r4rFrGrHris r, to_jsonablezIntVar.to_jsonables7J''II8888!%!7!7   r-rg) r4 str | NonerF int | NonerGrnrHrnr5r6r5r7r5r\r5zdict[str, Any]) r8r9r:r; itertoolscountrWr)rerjrlr>r?s@r,rrsr<#9??$L  (,       &    B < r-rct|trd}nt|trd}ny|jjt urt |d|d|dy)zXIf ``v`` is a SymInt or SymBool, validate it originates from the spec ShapeEnv. r rN: z= spec values must originate from spec IntVar / ShapeVar; got z backed by a different ShapeEnv.) isinstancer rr^rPrCr3)vwherekinds r,_validate_spec_symrzsk!V Aw vv244gRvE!A C  5r-cDeZdZdZ ddddd dfdZxZS)rzIndicates that a dimension size is dynamic and is >= 0. Subclass of `IntVar` with ``min=0`` by default. Example:: ShapeVar("batch") ShapeVar("batch", max=64) ShapeVar("batch", max=64, optimization_hint=32) NrrEcT|dkrtd|dt| ||||y)NrzBShapeVar requires min >= 0 (a shape dim is non-negative); got min=z+. Use IntVar(...) for unrestricted scalars.rE) ValueErrorr(r))r*r4rFrGrHr+s r,r)zShapeVar.__init__sE 7%JL  3CCTUr-rg) r4rmrFr\rGrnrHrnr5r6)r8r9r:r;r)r>r?s@r,rrs_   V(, V V V  V & V  V Vr-rc@eZdZdZd dZd dZd dZd dZd dZddZ y)ra~Per-dimension shape specification for a tensor. A list-like container of ``LeafIntSpec`` with length equal to the tensor's dim. Example:: B = ShapeVar("batch") TensorSpec([B, None]) # rank 2, dim 0 dynamic TensorSpec([B, 10]) # rank 2, dim 0 dynamic, dim 1 static=10 TensorSpec([B * 2 + 1, None]) # rank 2, dim 0 derived from B c t|D]T\}}|=t|ttfs't d|dt |j d|t|d|Vt||_ y)NzTensorSpec dim z: expected LeafIntSpec, got rurx) enumeratervr\r r3r2r8rzlist_specs)r*dimsids r,r)zTensorSpec.__init__s}dO ?DAq}ZC=%A%aS(DAw''(1%1 q/!(= >  ?*.d r-c |j|Srg)r)r*indexs r, __getitem__zTensorSpec.__getitem__&s{{5!!r-c,t|jSrg)lenrris r,__len__zTensorSpec.__len__)s4;;r-c,t|jSrg)iterrris r,__iter__zTensorSpec.__iter__,sDKK  r-cdg}t|jD]!\}}|jt|d|#dj |S)NzTensor:ru )rrrb_INDENTrc)r*linesrspecs r,rezTensorSpec.__repr__/sO  - 4GAt LLG9QCr$2 3 4yyr-c~d|jDcgc] }t|dr|jn|"c}dScc}w)Nrrl)r2r)rhasattrrlr*rs r,rlzTensorSpec.to_jsonable5sG !KK'.dM&B  "L  %:N)rzSequence[LeafIntSpec]r5r6)rr\r5rrp)r5zIterator[LeafIntSpec]rorq) r8r9r:r;r)rrrrerlr-r,rrs% 4" !  r-rcJeZdZdZd d dZd dZd dZddZddZddZ dd Z y)raSpec for any Python object's attributes. Constructor:: ObjectSpec({name: IntermediateSpec, ...}) Values may be leaves (``TensorSpec`` / ``IntVar`` / ``int`` / ``None``) or another ``ObjectSpec`` for recursion. Example:: ObjectSpec({"weight": TensorSpec([ShapeVar("h"), None])}) ObjectSpec({"inner": ObjectSpec({"weight": TensorSpec([ShapeVar("h")])})}) Nc8|rt||_yi|_yrg)dict_fields)r*fieldss r,r)zObjectSpec.__init__OsDJDL PR r-c||jvSrg)r)r*r4s r, __contains__zObjectSpec.__contains__Rst||##r-c,t|jSrg)rrris r,rzObjectSpec.__iter__UsDLL!!r-c,t|jSrg)rrris r,rzObjectSpec.__len__Xs4<<  r-c6|jjSrg)ritemsris r,rzObjectSpec.items[s||!!##r-c^dg}|jjD]}\}}t|}d|vrL|jtd|d|j D]}|jtdz|za|jtd|d|dj |S)Nz object_spec:r.:ru)rrreprrbr splitlinesrc)r*rr4r spec_reprlines r,rezObjectSpec.__repr__^s ,,,,. ?JD$T Iy  y$q12%0025DLL1t!345 y$r)=> ?yyr-c d|jjDcic]$\}}|t|dr|jn|&c}}dScc}}w)Nrrl)r2r)rrrrl)r*r4rs r,rlzObjectSpec.to_jsonablejsV #',,"4"4"6D$GD-,Hd&&(dR  s)Arg)rz"dict[str, IntermediateSpec] | Noner5r6)r4r&r5bool)r5z Iterator[str]rpr5rrorq r8r9r:r;r)rrrrrerlrr-r,rr?s+ S$"!$   r-rcReZdZdZ d d dZd dZd dZddZddZddZ dd Z y)raTSpec for a Python ``dict``-typed value. Constructor:: DictSpec({key: IntermediateSpec, ...}) Keys may be ``str`` or ``int``. Example:: DictSpec({"x": TensorSpec([ShapeVar("h"), None])}) DictSpec({"config": DictSpec({"batch": IntVar()})}) DictSpec({0: TensorSpec([ShapeVar("h"), None])}) Nc|r t|ni|_|jD]<}t|ttfrt dt |jd|y)Nz0DictSpec entries must have str or int keys, got ru)r_entriesrvr7r\r3r2r8)r*entriesks r,r)zDictSpec.__init__sd%DM"  Aa#s,FtAwGWGWFXXZ[\Z_` r-c||jvSrg)r)r*keys r,rzDictSpec.__contains__sdmm##r-c,t|jSrg)rrris r,rzDictSpec.__iter__sDMM""r-c,t|jSrgrrris r,rzDictSpec.__len__4==!!r-c6|jjSrg)rrris r,rzDictSpec.itemss}}""$$r-c^dg}|jjD]}\}}t|}d|vrL|jtd|d|j D]}|jtdz|za|jtd|d|dj |S)Nz dict_spec:r[]:r]: )rrrrbrrrc)r*rrrrrs r,rezDictSpec.__repr__s,,. AICT Iy  y#34%0025DLL1t!345 y#I;?@ Ayyr-c d|jjDcic]-\}}t|t|dr|j n|/c}}dScc}}w)Nrrlr2r)rrr7rrl)r*rrs r,rlzDictSpec.to_jsonables["&!4!4!6CCm0L$**,RVV  s2Arg)rz(dict[str | int, IntermediateSpec] | Noner5r6)rr&r5r)r5zIterator[str | int]rprrorqrrr-r,rrtsA  CG ?   $#"%   r-rc0eZdZdZddZddZd dZd dZy) raSpec for a Python ``list``- or ``tuple``-typed value. Per-position. The spec list may be shorter than the runtime sequence; any positions beyond ``len(self)`` are treated as fully static (i.e. equivalent to an unspecified slot). Constructor:: SeqSpec([IntermediateSpec, ...]) Example:: SeqSpec([TensorSpec([ShapeVar("h"), 10]), 1]) SeqSpec((TensorSpec([ShapeVar("a")]), TensorSpec([ShapeVar("b")]))) c$t||_yrg)rr)r*rs r,r)zSeqSpec.__init__s 04W  r-c,t|jSrgrris r,rzSeqSpec.__len__rr-cTdg}t|jD]}\}}t|}d|vrL|jtd|d|j D]}|jtdz|za|jtd|d|dj |S)Nz seq_spec:rrrrr)rrrrbrrrc)r*rrrrrs r,rezSeqSpec.__repr__s  / =GAtT Iy  y!B/0%0025DLL1t!345 y!C {;< =yyr-c~d|jDcgc] }t|dr|jn|"c}dScc}w)Nrrlr)rrrlrs r,rlzSeqSpec.to_jsonablesG!MM'.dM&B  "L  rN)rzSequence[IntermediateSpec]r5r6rprorq)r8r9r:r;r)rrerlrr-r,rrs >"   r-rrrrrParamsSpecValuec:eZdZdZdZdZ d d dZd dZd dZy) ra"Specification for the arguments of a compiled function. Describes the dynamic shape behavior for named arguments, ``*args``, and ``**kwargs`` of a ``torch.compile``-wrapped function. Takes a single dict keyed by parameter name, with two reserved sentinel keys for the variadic slots:: def func(x, n, *args, **kwargs): ... ParamsSpec( { "x": TensorSpec([ShapeVar("batch"), None]), "n": IntVar("seq"), "*args": [TensorSpec([ShapeVar("a")]), None], "**kwargs": { "foo": TensorSpec([ShapeVar("b"), None]), "bar": TensorSpec([ShapeVar("c"), None]), }, } ) Anything not expressed in ``ParamsSpec`` is STATIC.** z*argsz**kwargsNc Xi|_d|_d|_|y|jD]~\}}||jk(rvt |t s.td|jdt|jt|D]\}}t|d|d|dt ||_||jk(r|t |ts.td|jdt|j|jD]\}}t|d|d|dt||_|jdr)td |d |jd |jd t|d |dtt ||j|<y)Nz ParamsSpec z) value must be a list of leaf specs, got z ParamsSpec 'z'[]rz) value must be a dict of leaf specs, got *zUnknown sentinel key z in ParamsSpec; only z and z are reservedz ParamsSpec[) _named_args_varargs_varkwr _VARARGS_KEYrvrr}r2r8rrz _VARKW_KEYrr0rr)r*paramsrvaluerrwrs r,r)zParamsSpec.__init__s9;7; :> >  ,,. FJCd'''!%.$%d&7&7%:;..25k.B.B-CE&e,LDAq&q,se2aS0JKL $U  '!%.$%doo%89..25k.B.B-CE"KKMNDAq&q,se2aU!0LMN"5k $ +C72G((+50C=R #5+cWA0FG(,-=u(E  %5 Fr-c@tfdddfd |jjDcgc]\}}||}}}|jSdj fdt |jD}|j |jd||jXdj fd|jjD}|j |jd|dj |Scc}}w) NcTdjfd|jDS)Nrc3(K|] }|z ywrgr).0rprefixs r, z=ParamsSpec.__repr__.._indent_lines..>sItVd]Is)rcr)textrs `r, _indent_linesz*ParamsSpec.__repr__.._indent_lines=s99It7HII Ir-cJt|}d|vr |d|S|d|S)Nr: ru)r)rrv_reprrs r, _entry_reprz(ParamsSpec.__repr__.._entry_repr@s;%[Fv~c-"7!899U"VH% %r-rc3HK|]\}}t||ywrg)r7)rrrwrs r,rz&ParamsSpec.__repr__..Hs%+/1a CFA&s"rc36K|]\}}||ywrgr)rrrwrs r,rz&ParamsSpec.__repr__..MsPDAqk!Q/Ps)rr7rr7r5r7)rr7rrr5r7) rrrrrcrrbrrr)r*rrwrinnerrrs @@r,rezParamsSpec.__repr__<s3: J & 04/?/?/E/E/GHtq!Q"HH == $II3 10, A * 2 == B], ) ``globals`` is reserved for future use and will raise ``NotImplementedError`` if set. N)globals assumptionsc d|_g|_| tdt|tr t |}n3|1t|t s!t dt|j||_|wt|D]h\}}t|ts't d|dt|jd|t|d|d|jj|jyy)Nz'ShapesSpec.globals is not supported yetz3shapes_spec must be ParamsSpec, dict, or None, got zShapesSpec.assumptions[z]: expected SymBool, got rurr) _params _assumptionsNotImplementedErrorrvrrr3r2r8rrrzrb)r*rrrras r,r)zShapesSpec.__init__}s+/ +-  %&OP P fd #'F   6:(FF|,,-/   "!+. ,1!!W-#1!5#Aw//01%9#1.EaS,JK!!((+ , #r-c,t|jSrg)rrris r,rzShapesSpec.assumptionssD%%&&r-cdg}|j]|jtdt|j}|j D]}|jtdz|z|j rM|jtd|j D]&}|jtdzt|z(dj |S)Nz shapes_spec:zparams:rz assumptions:r)rrbrrrrrc)r*r param_reprrrs r,rezShapesSpec.__repr__s  << # LLG9G, -dll+J"--/ 1 Wq[4/0 1    LLG9L1 2&& 4 Wq[4723 4yyr-cd|jdn|jj|jDcgc] }t|c}dScc}w)Nr)r2rr)rrlrr)r*rs r,rlzShapesSpec.to_jsonablesH "ll2d 8P8P8R-1->->?DG?  @sArg)rz.ParamsSpec | dict[str, ParamsSpecValue] | NonerrrzSequence[SymBool] | Noner5r6)r5z list[SymBool]rorq) r8r9r:r;r)propertyrrerlrr-r,rrfse0BF",04 ",>", ", . ",  ",H''   r-r)r5r )rwrrxr7r5r6),r; __future__rrr threadingtypingrrrrrYtorchrr collections.abcr r rBr __all__rrrr=LockrArCrrzrrrrrr\rrrrrr7rrrrr-r,rs< #66 !2> "  $(' &y~~'>BZ VZ z "VvV8. . b2 2 j; ; |* * p &3.5 Y5!J.).'3h>H)Ht,--S:J5J0KK i i XQ Q r-