^jjUddlZddlmZmZddlmZmZddlmZm Z ddl Z gdZ ejeZedZe dZeeefZeeefZeGdd ZeeeeegeefZeGd d ZeZeed <iaeeeefe j>j@fed <iZ!ee"efed<eeeefeefZ#ia$e#ed<eeeeeeffZ%ia&e%ed<ia'e%ed<ia(e%ed<da)e*ed<e+hdZ,e+eed<deeezdzdeeezdzdeeezdzde-eeeffdZ.dddeddfdZ/iZ0eee j>j@fed<e-Z1e-eed <iZ2eeeefe j>j@fed!<d"ede j>j@fd#Z3d$ede j>j@fd%Z4d&ed$ed'eddfd(Z5d&ed$eddfd)Z6d&edd*fd+Z7d&edefd,Z8d-ed&eddfd.Z9d/e j>j@d0ed$eddfd1Z:d2eeezdzdeefd3Z;dddd4d5eeezdzd6eeezdzd7eeezdzddfd8Zedeefd?Z>d>ed&ed$ed@eeefddf dAZ?dBe"dCe"defdDZ@dddEdFedGed&ed$edHedzdIedJedKeddfdLZAddMdNeedOeedPedefdQZBddRd&ed$edSeedTedzddf dUZCddddVdWedXe-eeefdzdYedTedzddf dZZD dgd[edee"effd\ZEddRd&ed$edSeedTedzddf d]ZFdhd^ZGdd_d`edYeddfdaZHdd_dbeeeegefdYeddfdcZIdd_ddeeegefdeedYeddfdfZJy)iN)CallableIterable) dataclassfield) ParamSpecTypeVar)UserOrderingFnregister_op_override!reorder_graphs_from_user_functionreenable_op_overridesderegister_op_overridesget_dsl_operationsnative_decomp_tablePRcjeZdZUdZeed<eed<eed<eed<eed<eed<dZe ed <d Z e ed <y ) _OverrideNodezTrack function override data.dsl_name op_symbol dispatch_keycond_fnimpl_fnnode_idFunconditional_overrideTactiveN) __name__ __module__ __qualname____doc__str__annotations__ _OpCondFn _OpImplFnrboolra/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/_native/registry.pyrrs='MN   L#(D(FDr&rc eZdZUdZeeZeeed<eeZ eeed<eeZ eeed<de de fdZ dd eeezd zd eeezd zd eeezd zd e dd f dZdeeeeffdZdefdZy ) _FilterStatez+Manages filtering state for override nodes.)default_factory _dsl_names _op_symbols_dispatch_keysnodereturnc|j|jvry|j|jvry|j|j vryy)z Check if a node is enabled based on current filter state. Args: node: The override node to check Returns: bool: True if the node should be enabled, False if filtered out FT)rr+rr,rr-)selfr.s r' check_enabledz_FilterState.check_enabled9sF ==DOO + >>T-- -    3 3 3r& dsl_namesN op_symbols dispatch_keys remove_keysc|rv|xjtt|zc_|xjtt|zc_|xjtt|zc_y|xjtt|zc_|xjtt|zc_|xjtt|zc_y)a Update filter sets as (current | new) or (current ~ new). Args: dsl_names: DSL names to add/remove from filter op_symbols: Operation symbols to add/remove from filter dispatch_keys: Dispatch keys to add/remove from filter remove_keys: If True, remove keys from filter; if False, add them Note: Uses set.discard as it doesn't raise an exception if the element wasn't in the set to begin with. N)r+set_resolve_iterabler,r-)r1r3r4r5r6s r'updatez_FilterState.updateNs(  OOs#4Y#?@ @O   $5j$A B B    3'8'G#H H  OOs#4Y#?@ @O   $5j$A B B    3'8'G#H H r&cXt|j|j|jS)z Build a set of dictionary keys based on the current filter state. Returns: set[tuple[str, str]]: Set of (op_symbol, dispatch_key) tuples )_build_key_setr+r,r-)r1s r'build_disable_key_setz"_FilterState.build_disable_key_setks* OO        r&c(d}|dz }|dz }t|jD]\}}|d|d|dz }|dz }t|jD]\}}|d|d|dz }|dz }t|jD]\}}|d|d|dz }|S) z-Return string representation of filter state.zFilter State: z === DSL: ===  z:  z === OP SYMBOL: === z === DISPATCH KEYS: === ) enumerater+r,r-)r1sidslopkeys r'__str__z_FilterState.__str__xs   0 %FAs 4s"SE$ $A % %%t//0 $EAr 4s"RD# #A $ )) 3 34 %FAs 4s"SE$ $A %r&F)rrrrrr8r+r r!r,r-rr$r2rr:tupler=rHr%r&r'r)r)1s5 5JC5!#6KS6$S9NCH9-D4" I#&-I(3-'$.IXc]*T1 I  I  I:  s5c?';  r&r) _filter_state_libs_native_decomp_overrides_graphs_dsl_name_to_lib_graph_dispatch_key_to_lib_graph_op_symbol_to_lib_graph_node_id_counter>MetaCompositeExplicitAutogradCompositeImplicitAutograd_DISALLOWED_DISPATCH_KEYSr3r4r5r/ctdtttzdzdtddffd }||t||t ||t S)aK Build a set of dictionary keys based on filter criteria. Args: dsl_names: DSL names to include in key set op_symbols: Operation symbols to include in key set dispatch_keys: Dispatch keys to include in key set Returns: set[tuple[str, str]]: Set of (op_symbol, dispatch_key) tuples entriesNgraph_lib_dictr/cjt|}|D]"}||vs||D]}j|$y)z;Helper to add matching keys from graph_lib_dict to key_set.N)r9add)rXrYresolved_entriesentryrGkey_sets r'_append_to_setz&_build_key_set.._append_to_setsG-W5% %E&)%0%CKK$% %r&)r8r r _MappingTyperOrQrP)r3r4r5r_r^s @r'r<r<s_ %(EG %x}$t+ %=I %  %945:67="<= Nr&F)print_inactiverac$tjD]y\\}}}td|d|t|D]R\}}|js|sd|d|j d|j }|r|d|jz }t|T{y)z~ Print all override graphs for debugging purposes. Args: print_inactive: Whether to print inactive nodes zop=z, key=r@z: node.dsl_name=z, node.unconditional_override=z node.active=N)rNitemsprintrBrrr)rarFrG node_listrDr.rCs r'_print_override_graphsrfs!(   S9 Wv + GAt{{ns"3T]]$44St7R7R6TU!>T[[N++Aa   r& _def_libs_defined_native_ops_aten_override_libs namespacecr|tvr'tjj|dt|<t|S)NFRAGMENT)rgtorchlibraryLibrary)rjs r'_get_def_libraryrps0 !$}}44Y K ) Y r&rc|d|f}|tvr(tjjdd|t|<t|S)z Get or create the _native IMPL library for a given dispatch key. One library per dispatch key is shared across all overridden ops. _nativeIMPL)rLrmrnro)rrGs r'_get_or_create_libraryrt s< l #C %]]**9flKc :r&rkernelc||f}tj|d}||jtjj dd|}|j |||d|t|<y)a Install (or replace) an aten kernel at (op_symbol, dispatch_key). Creates a fresh Library per (op, key) so we can tear down just this one override via `_destroy_aten_override` without affecting any other override at the same dispatch key. NatenrsT) with_keyset)ripop_destroyrmrnroimpl)rrrurGexistinglibs r'_install_aten_overrider~sk l #C#&&sD1H --   =CHHY $H?"r&cZtj||fd}||jyy)zATear down the aten override at (op_symbol, dispatch_key), if any.N)riryrz)rrr}s r'_destroy_aten_overrider.s,  ! !9l";T BC  r&ztorch._ops.OpOverload | Nonec|jd\}}}|xsd} ttjj|}t||S#t $rYywxYw)u4 Resolve `op_symbol` to a concrete OpOverload on `torch.ops.aten`. Accepts bare names ("bmm" → aten.bmm.default) and overload-qualified names ("add_.Tensor" → aten.add_.Tensor). Returns None if the op is not registered (e.g. a test-only op_symbol that never hit the C++ dispatcher). .defaultN) partitiongetattrrmopsrwAttributeError)rname_ overload_namepackets r'_resolve_aten_overloadr5s]'005D!]!.YM.v}-- s/A AAct|}|td|t|j}|j dd\}}|j dd\}}d|S)uReturn the schema of at:: with the `aten::` prefix stripped. Accepts bare names ("bmm" → aten.bmm.default) and overload-qualified names ("add_.Tensor" → aten.add_.Tensor). z aten op not found for op_symbol=z::()rrr _schemasplit)roverloadrCrrestargss r'_aten_schema_tailrFsk &i0H? }MNN H  AggdAGAtjja GAt tf:r&rc|tvrytdj|t|t |}t j jd||ddtj|y)Nrrz _native::)_aten_overloadc||i|SNr%)rrkwargss r'z(_define_native_op_once..dsn ? ? r&) rhrpdefinerrrmrn register_faker[)rr aten_overloads r'_define_native_op_oncerVsu  ""Y&&$0A)0L/M'NO+95M MM)D6 23%2 D!r&r}r.c|js&td|jd|jdt |j|j|j |j|j |ddy)z Register a single node implementation with the library. Args: lib: The torch.library.Library instance node: The override node to register dispatch_key: The dispatch key for registration zJ_OverrideNode must have a non-empty node_id before registration (dsl_name=z , op_symbol=)FT)rxallow_overrideN)r ValueErrorrrrr{r)r}r.rs r'_register_node_implrrsv <<)dnn5Gq J  4<<8HH    r&iterablecT|gSt|trt|tr|fS|S)z Resolve various input types to a consistent iterable of strings. Args: iterable: String, iterable of strings, or None Returns: Iterable[str]: Consistent iterable output ) isinstancerr )rs r'r9r9s/ h )Z#-F{ Or&)enable_dsl_namesenable_op_symbolsenable_dispatch_keysrrrctjd|||tj|||dt |||}|D]*}|\}}|t vst ||t |t,y)as Re-enable overrides by removing them from filter state and reregistering. Args: enable_dsl_names: DSL names to re-enable enable_op_symbols: Operation symbols to re-enable enable_dispatch_keys: Dispatch keys to re-enable Note: This function uses reverse filter state management (removing from filters to enable). z>Re-registering ops by dsl: %s, op_symbol: %s, dispatch_key: %sT)r6 filter_stateN)loginforKr:r<rN_register_overrides_from_graph)rrrr^rGrrs r'r r s$HHH  %3%G"% < '> +<M  r&)disable_dsl_namesdisable_op_symbolsdisable_dispatch_keysrrrctjd|||tj|||tj }|D]*}|\}}|t vst ||t |t,y)ag De-register overrides by updating filter state and reregistering graphs. Args: disable_dsl_names: DSL names to disable disable_op_symbols: Operation symbols to disable disable_dispatch_keys: Dispatch keys to disable Note: This function uses filter state management to selectively disable operations. z>De-registering ops by dsl: %s, op_symbol: %s, dispatch_key: %srN)rrrKr:r=rN_cleanup_and_reregister_graph)rrrr^rGrrs r'r r s}$HHH *,>@UV%2$G$G$IG "% < '> ) *   r&rct}tjD]1\\}}}|D]$}|j|k(s|j |13t |S)zGet list of operations registered by a specific DSL. Args: dsl_name: Name of the DSL to query. Returns: Sorted list of operation names registered by the DSL. )r8rNrcrr[sorted)r operationsrrnodesr.s r'rrs`J!(A D}}(y)  * r&rGcdttttttffdtdtttfddfd}|t|||t |||t ||y)z Update the registration mapping dictionaries. Args: dsl_name: The DSL name op_symbol: The operation symbol dispatch_key: The dispatch key key: The dictionary key tuple registrationsymbolrGr/Nc^|j|}| |g}|||<y|j|y)z;Helper to add key to registration list or create new entry.N)getappend)rrrG entry_lists r'_get_new_entry_or_appendz;_update_registration_maps.._get_new_entry_or_append%s9 "%%f-  J#-L   c "r&)dictr listrJrOrQrP)rrrrGrs r'_update_registration_mapsrss& #3U38_ 556 # #38_ #  #3XsC4iE7sKr&rrcy)NTr%)rrs r' _always_truer8s r&)allow_multiple_overriderbackend lib_symbolcondr{rrc ||dk7rtd|d|tvrtd|d||s tdt}||f}tj |g} |j dd } | d |d t } t d z a| jt||||||| | t|<t|||| y) a Register a passed override function to the dispatcher. Actually a graph-building operation; real registration happens later. Args: backend: The backend name (DSL name) lib_symbol: Library you're overriding symbols in (must be "aten") op_symbol: Name of the operation you're overriding dispatch_key: Dispatch key to override cond: Predicate choosing whether `impl` applies to a given call. May be None if `unconditional_override=True`. impl: Implementation function for the override allow_multiple_override: Allow overriding an existing override unconditional_override: Implementation doesn't have a fallback and doesn't require torch.DispatchKeySet as the first argument. When True, a trivially-True predicate is supplied for the router if `cond` is None. Raises: ValueError: If lib_symbol is not "aten", if dispatch_key is in _DISALLOWED_DISPATCH_KEYS (Meta / CompositeImplicitAutograd / CompositeExplicitAutograd), or if cond is None without unconditional_override=True. rwz.Unsupported lib_symbol (must be "aten", got: ""z dispatch_key=z is not supported. Overrides must be installed at a backend key (e.g. CPU, CUDA, XPU); the router's fake kernel redispatches to aten and would recurse otherwise.Nz8cond must be provided unless unconditional_override=Truerrr)rrrrrrr)rG) rrVrrNrreplacerRrrr) rrrrrr{rrrGop_graph _sanitizedrs r'r r <sHVI*UVWXX00L+,G H   |%WX X l #C{{3#H ""3,J AgYa(8'9:G OO%#9  GCLgy,CHr&force_reregisteroriginal_graph new_graphrc|ry||k7S)aG Determine if a graph needs reregistration based on changes. Args: original_graph: The original graph before modification new_graph: The graph after modification force_reregister: If True, always reregister regardless of changes Returns: bool: True if reregistration is needed Tr%)rrrs r'_should_reregister_graphrs" Y &&r&rgraphrc"t||||y)a Reregister a graph's routes from scratch. Used by reorder / deregister / reenable. Libraries are intentionally long-lived singletons; we rebuild the per-op router closure here. Args: op_symbol: The operation symbol dispatch_key: The dispatch key graph: The graph to register filter_state: Optional filter state for conditional registration rN)r)rrrrs r'rrs&# ! r&)keys_to_processreregister_overridesrtransformation_fnrrcB||nttj}t|D]\}}||ftvrtt||f} ||||}t|ts,tjdt|j||l|t||f<|szt||dst!|||| y#tt t tf$rtjd||dYt$rtjd||YwxYw) a Apply a transformation function to graphs and optionally reregister. This is the core pattern used by reorder_graphs_from_user_function and can be reused for other graph transformation operations. Args: transformation_fn: Function to transform each graph keys_to_process: Keys to process, or None for all graphs reregister_overrides: Whether to reregister changed graphs filter_state: Optional filter state for conditional registration Note: If transformation_fn raises an exception for a specific graph, that graph will be skipped and processing will continue with remaining graphs. NAGraph transformation failed for %s/%s. Preserving original graph.Texc_infoNUnexpected error in graph transformation for %s/%s. Preserving original graph.zbGraph transformation returned invalid type %s for %s/%s. Expected list. Preserving original graph.Frr)r8rNkeysr TypeErrorrr RuntimeErrorrwarning Exception exceptionrtyperrr) rrrr target_keysrrrrs r'_apply_graph_transformationrs=6+6C RI$)T* KKtY((   .7L)* $< I%  *)  U/:~|D  KKS     MM`    s C2D<DDoverrides_onlycf|ri}nddlm}t|}|jt|S)a Return a decomposition table suitable for passing to ``ExportedProgram.run_decompositions`` or to Inductor / Dynamo as a decomposition set for ``torch.compile``. This is the canonical way to apply native overrides outside of eager. Callers opt in explicitly -- the registry does **not** install into any global compile/export decomp table on its own. This preserves three properties: * ``import torch._native`` doesn't transitively import inductor / dynamo / triton. * No other consumer (ONNX, tests, third-party backends) accidentally picks up our overrides. * Scoping is up to the caller: pass the table only where routing is desired. By default, the returned table contains ``torch.export.default_decompositions()`` with native-registered overrides layered on top -- matching typical "run the usual decompositions, plus my overrides" intent. The override entries win over any same-op default, because they're merged last. Example: ep = torch.export.export(model, args) ep = ep.run_decompositions( torch._native.registry.native_decomp_table() ) Args: overrides_only: If True, return only the native-registered overrides and no default aten decompositions. Useful for inspection / debugging, or when composing tables manually. Returns: A dict mapping ``OpOverload`` to decomposition callable. r)default_decompositions) torch.exportrrr:rM)rtablers r'rrs2R(* 8+-. LL)* Lr&c t|}g |D]\}d}|r|j|}|r._dispatchsv( NOD) // .eager_routers;4G Y '9''@@@ @ r&c2||d}|urtS|S)NTr)NotImplemented)rrrrrs r'compile_routerz6_register_overrides_from_graph..compile_routers$4F Y ! ! r&)rtr2rrrrrrrrMryrmrn get_kernelobjectr$r~)rrrrr}r.enablerfallback_kernelrrrrrs @@@r'rrFs ! .C-/I    !//5F  T< 8   dllDLL9 :DKDK  &i0H 9l3    $ ( (4 8mm.. {/C\ROI  /> 9lLA -; *r&c`tjD]\}}|\}}t|||y)zO Perform all registration calls from previously-built override graphs. N)rNrcr)rGrrrs r'_register_all_overridesrs8mmo U"% <&     r&)rfnct||y)a Reorder override graphs using a user-provided ordering function. Args: fn: User-provided function that takes (op_symbol, dispatch_key, graph) and returns a reordered graph reregister_overrides: Whether to reregister graphs that have changed Note: This function uses the common graph transformation pattern and can serve as an example for other graph manipulation operations. rrN)r)rrs r'r r s" 1r& filter_fnc xdtdtdttdttffd }t||y)a Apply a filter function to remove nodes from graphs. This is a convenience function that uses the graph transformation pattern to filter out unwanted nodes. Args: filter_fn: Function that takes (op_symbol, dispatch_key, node) and returns True to keep the node, False to remove it reregister_overrides: Whether to reregister modified graphs Example: # Remove all nodes with "deprecated" in the DSL name _apply_graph_filter( lambda op, dk, node: "deprecated" not in node.dsl_name, reregister_overrides=True ) Note: If filter_fn raises an exception for a specific graph, the original graph will be preserved and processing will continue. rrrr/c |Dcgc]}|||s|c}Scc}w#ttttf$rtj d||d|cYSt $rtjd|||cYSwxYw)z-Apply filter_fn to graph with error handling.rTrrrrrrrrrr)rrrr.r s r'filtering_transformationz5_apply_graph_filter..filtering_transformations %*WTi .conditional_transformation,s ))\BN"  "9lEBB" G:~|D  KKS   L  MM`  L  z><H  W !     d    s,  A84A5!A54A584C.!CCrNr)rrrrs`` r'_apply_selective_reorderingrsC<))&))26}2E) m )V 41r&rI)r/N)Kloggingcollections.abcrr dataclassesrrtypingrr torch.libraryrm__all__ getLoggerrrrrr$r"r#rr rr r)rKr!rLrrJrnrorMr _GraphsTyperNr`rOrPrQrRint frozensetrVr8r<rfrgrhrirprtr~rrrrrr9r r rrrr rrrrrrr rrr%r&r'r!s.(% g! cN CL QW  QTN   "3T-%894 ;NNO TT Tp+n |,79tE#s(OU]]22 238.46$vx/055c?D$778 CeCHo../ (* )+-L-(** #-6F-9S> !Xc]"T)!hsm#d*!#&-! sCx !H6;dt(/1 4U]]*** +0 #SX% EGT%S/5==+@+@@AF   (=(=  1F1F #c##h#SW#*cc.L"    """"8   &3CF 4hsm 3d :x}*040437 2DIo,2T#Y-2S /D0 2  2n151548 )T#Y-)d3i$.)c?T1 )  )Xc$#L#L#L#L sCx #L  #LL&T%*#(MI MIMIMI MI d  MI  MI"MI!MI MIh# '''M"' '  ':)-    %   <48!&(, N%NsCx)D0N N % N  Nd!33 &( 3v)- h<h<h<  h< % h<  h