^j,ddlmZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl m Z mZddlmZddlmZmZddlmZmZmZmZmZddlmZmZmZddlZdd lmZddl Z dd l!m"Z"dd l#m$Z$m%Z%d Z&e"Z'dd Z(ddZ)dZ*dZ+Gdde jXjZj\Z.dZ/d dZ0e GddZ1dZ2dZ3GddZ4GddZ5 d!dZ6y)") annotationsN) dataclassfield)Path)cached_propertypartial)DictTupleListOptionalAny)_bench_cuda_graph_l2_rotate_clone_l2_rotate_inputs_pick_l2_rotate_count)Tensor) __version__)make_epilogue_source_markerserialize_worker_valuetorch_vendor_quackcrt|ttfryt|}t|trd|vryy)aZReturn True for failures that should invalidate one config only. Autotune candidates can be structurally invalid for a particular epilogue and shape (for example, a fragment layout that cannot represent a requested local-reduce group). Those should get an infinite timing while unexpected programming errors still propagate. Tz%expected reshaped size to be the sameF) isinstance RuntimeError MemoryErrorstr ValueError)emsgs h/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/_vendor/quack/autotuner.py$_is_expected_autotune_config_failurer &s8!lK01 a&C!Z %LPS%S ctjjrtjjsytjj }t j jd}|@|jdDcgc]}|j}}|t|kr||Syt|Scc}w)aReturn the physical CUDA device identifier for the current process. Maps the logical ``torch.cuda.current_device()`` index through ``CUDA_VISIBLE_DEVICES`` (if set) so the result is valid as a standalone ``CUDA_VISIBLE_DEVICES`` value (handles integer IDs, GPU UUIDs, and MIG IDs). Returns ``None`` if CUDA is not initialized or the device cannot be determined. NCUDA_VISIBLE_DEVICES,) torchcuda is_availableis_initializedcurrent_deviceosenvirongetsplitstriplenr)logical_deviceparent_visibledvisible_devicess r_get_current_cuda_devicer46s JJ # # %%***C*C*EZZ..0NZZ^^$:;N!.<.B.B3.GH1779HH C0 0">2 2 ~  IsCc|tjtjdt j S)N_HOME)r*getenv PACKAGE_NAMEupperrhomer!r get_home_dirr<Ms* 99 **,-U3TYY[ AAr!cbtjjtdtdS)N.cache)r*pathjoinr<r8r;r!rdefault_cache_dirrBQs" 77<< !L>(:G DDr!ceZdZfdZxZS)FileCacheManagerct||tjtj ddj xs t|_|jrtjj|j|j|_tjj|jd|_ tj|jdytd)N _CACHE_DIRlockT)exist_okz$Could not create or locate cache dir)super__init__r*r7r8r9r.rB cache_dirr@rAkey lock_pathmakedirsr)selfrM __class__s rrKzFileCacheManager.__init__Vs  II++-.j92 > D D F ]J[J]  >>WW\\$..$((CDNWW\\$..&ADN KK 6EF Fr!)__name__ __module__ __qualname__rK __classcell__)rQs@rrDrDUs G Gr!rDctjtj|j dj dS)Nutf-8=)base64 b32encodebytesfromhexdecoderstriprMs r_base32r`cs2   EMM#. / 6 6w ? F Fs KKr!ctjdddtj}tjj |dz }t j }t j |z |krOt dD]}||z} tjj t j |z |krNyy)zSaturate the GPU to reach thermal steady-state before benchmarking. Without this, the first autotuning config gets artificially good numbers because the GPU hasn't been power-throttled yet. ir&)devicedtypeidN)r%randnbfloat16r& synchronizetimerange) duration_msatargett0_s r _gpu_warmuprohs  D$vU^^DA JJ 4 F B ))+ V #s AAA    ))+ V #r!ceZdZUdZeeZded<eeZded<ee Z ded<ee Z d ed <ee jZd ed <dddZddZddZy )_PrecompileHandlea&Per-config completion handle returned by :meth:`Autotuner._precompile`. Empty / no-op when the subprocess pool was skipped (cache hit, disabled, only 1 config). Calling ``wait_for(i)`` on a missing index returns immediately as a no-op, so the bench loop can iterate uniformly. )default_factoryzDict[int, threading.Event]eventszDict[int, str]failures List[Any]_workerszList[threading.Thread]_reader_threadszthreading.Lock_lockNcd|jj|}||j|yy)aBlock until ``config_idx``'s compile completes (or failed). No-op for indices that were never registered (e.g. the entire handle is empty because precompile was skipped). The bench loop relies on this to call ``wait_for`` unconditionally. Ntimeout)rsr,wait)rP config_idxr{evts rwait_forz_PrecompileHandle.wait_fors.kkooj) ? HHWH % r!cb|j5||jvcdddS#1swYyxYw)a/True if the subprocess pool failed to compile this config. Use after :meth:`wait_for` returns. Failed configs need an in-process jit_cache warm before benchmarking to avoid contaminating the timing measurement with compile cost (see :meth:`Autotuner._bench_with_warm`). N)rxrt)rPr}s r is_failedz_PrecompileHandle.is_faileds,ZZ /. / / /s%.c|jD]9} |j&|j|jj;|jD]} |j d|jD]}|jdy#tt f$rYwxYw#tj$r@|j |j dn#tj$rYnwxYwYwxYw)zClose stdins, wait for workers and reader threads to exit. Idempotent. Safe to call even if ``__init__`` failed midway through spawning workers. Ng@rz?) rvpollstdincloseBrokenPipeErrorOSErrorr| subprocessTimeoutExpiredkillrwrA)rPwts rshutdownz_PrecompileHandle.shutdowns  A 668#(;GGMMO   A s# %% A FF3F  $W-   ,, FF3F'!00  sG6BB&B#"B#&#C9 CC9C30C92C33C98C9N)r}intr{zOptional[float]returnNone)r}rrbool)rr)rRrSrT__doc__rdictrs__annotations__rtlistrvrw threadingLockrxrrrr;r!rrqrqsi*/t)DF &D$T:Hn:5Hi5.3D.IO+I!)..AE>A &/ r!rqc t|D]\}} t|j}|j 5|N||dD];}|jj|d|j|j= dddyt|tr |jdr||j|<|j|jdddy#ttt f$rd}YwxYw#1swYxYw)aDrain replies from one worker's stdout and signal per-config events. Replies arrive in dispatch order (length-prefixed pickled strings: ``"OK"`` or ``"ERR:..."``). On EOF (= worker crashed), every config this worker still had pending is marked failed and its event set so waiters don't deadlock. Runs in its own daemon thread so the bench loop can interleave waiting with kernel launches. Nzworker crashed during compilezERR:) enumerate_recv_from_workerstdoutrEOFErrorConnectionResetErrorrxrt setdefaultrssetrr startswith)handleworkerconfig_indicesposr}r remaining_idxs r_reader_thread_mainrs%^4,Z !&--0A\\ ,y&4CD%97MOO..}>]^MM-04467 , ,!S!all6&:./ + MM* % ) ) + , , , +?@ A  , ,s$CAC9A C9C65C69D cddl}ddl}|jd}t|dkry|j d|d}|dk(ry|j|}t||kry|j |S)u Read a length-prefixed pickled message. Returns None on EOF or partial body. Mirrors ``_compile_worker._recv`` so the parent and worker speak the same protocol. Kept private to this module rather than re-imported from the worker because the worker module isn't safe to import in the parent (it pushes ``_COMPILE_ONLY_DEPTH`` at module load). Returns ``None`` on both: * clean EOF (header shorter than 4 bytes — pipe write side closed before writing the header); * truncated body (header read OK, but the body read returns fewer bytes than the declared length — the worker was SIGKILL'd by its parent-death watchdog, or by OOM-killer, *between* writing the header and writing the body). Without the truncated-body check, ``pickle.loads`` raises ``UnpicklingError`` on a short body. That exception would propagate out of the reader thread, leaving the per-config events unset and deadlocking every ``wait_for(i)`` waiter in :meth:`Autotuner.__call__`'s ``benchmark()`` closure. rN._pre_hook9s1NRN`N`&adtVD\-?-?-A'A&a#&as4cxjD]#}||jj|%i_yr)rcopy_r)r exceptionrrPs r _post_hookz&Autotuner.__init__.._post_hookBs? ..BD4L&&t':':4'@AB&(#r!r perf_modeltop_kearly_config_prune)AutotuneConfigconfigsinspect signaturekeysr?r parameters arg_namesr*r7r8r9 cache_resultsrr/pre_hook post_hookr configs_top_krr,fn _do_bench) rPrrMrrprune_configs_bydo_benchrrrrs ` rrKzAutotuner.__init__sr *,-DL"DL%%b)  24 i22779:  _RYY,*<*<*>)??P'QSWX\__    $!%m!4D  t!! "Q & b&DM DM t!! "Q & ) (DN!DN "& .22<QDO!1!5!5gt?Q?Q!RD &6&:&:$d&=&='D #!r!ct|trd}|jdk(r|jjsk|j j jjj}tj|j}dt|jtd|jDt!|j|fSt|trtd|DSt|t"r#|Dcgc]}t$j'|c}St|t(r)tt+d|j-DS|Scc}w)Nrrc3,K|] }|dvr|ndyw)rrNr;).0ss r z%Autotuner._key_arg..bsF!1;aA-Fsc3FK|]}tj|ywrr_key_arg)rvs rrz%Autotuner._key_arg..gs>1++A.>s!c3PK|]\}}|tj|f ywrr)rkrs rrz%Autotuner._key_arg..ks$Utq!I$6$6q$9 :Us$&)rrndimrcis_floating_pointdetachcpu contiguousnumpytobyteshashlibsha256 hexdigesttupleshapestriderrrrrsorteditems)value value_key value_bytesrs rrzAutotuner._key_argXs eV $IzzQu{{'D'D#lln002==?EEGOOQ #NN;7AAC ekk"Fu||~FFEKK    eU #>>> > eT "389aI&&q)9 9 eT "Uu{{}UUV V :s$E>c h|jDcic]\}}||jvs||}}}|jDcgc]$}||vst|j ||&}}|jD]\}}t |t s|jt|j|jt|jD cgc] } | dvr| nd c} |jt|jt|Scc}}wcc}wcc} w)Nrr) rrrrrrrappendrrrcr) rPall_argsrr_argsrrMrnargrs r_make_cache_keyzAutotuner._make_cache_keyns$,NN$4L&1aT^^8KALL;?99V4PU s4==t-.VVkkm +FAs#v& 3syy>* 3NAQ&[a 7NOP 3syy>*  + SzMV OsD$D$ D* D*D/c~|j&ttjjddS|jS)N)warmuprep)rrtritontestingrrPs rrzAutotuner.do_benchxs/ >> !6>>221"E E~~r!c tddlm}|s tStt |t t jdd}|dkr tStj} t|fi|dj}|j|i|tj|z dkr tSt jtjddd k(}|rtd t |d |d ddlddlfd } |D cgc] } t%| } } |jj&} |jj(} |j+d}|j+d}t%t|}||t-|j.||d<t j0j3}t5}|||d<t7t j8|d<g}t;|D]}t=j>t@jBddgt<jDt<jD|st<jFnd|}tI|jJ}|dk7r|jM|jO||s tSt}||_(|Dcgc]}g}}tS|D]s\}}|t |z} | ||jT| | | ||jd||jO|t[j\|j^|<utS|D]\\}}||s t[jdtfd|||||fd}|ji|jjjO|^|S#t$rY/wxYwcc} wcc}w#tVtXf$rQt[j\|j^|<|j^|jad|jb|<Y[wxYw)uPre-compile all configs in parallel subprocesses to populate .o cache. cute.compile() is not thread-safe (MLIR thread-local state) and fork after CUDA init causes segfaults. So we spawn persistent subprocess workers: each has its own CUDA context, creates FakeTensors matching the parent's tensor metadata, and compiles with COMPILE_ONLY=True. Workers stay alive to amortize import overhead across multiple configs. The parent then loads instantly from the .o cache during benchmarking. Returns a :class:`_PrecompileHandle` whose ``wait_for(i)`` blocks until config ``i``'s compile completes; the bench loop interleaves ``wait_for`` + benchmark to overlap the remaining compiles with GPU work. Workers are equipped with a parent-death watchdog (:func:`quack._compile_worker._install_parent_watchdog`) so an orphaned worker self-terminates within ~60 s instead of lingering on long-lived self-hosted CI runners. The returned handle is also valid (empty but functional) when precompilation is skipped — cache disabled, only 1 config, the first-config warm-cache shortcut hit. Callers always call ``wait_for`` + ``shutdown``; the no-op handle makes both fast. r) CACHE_ENABLEDQUACK_COMPILE_WORKERS8r?_PRINT_AUTOTUNINGNrzPre-compiling z configs with z workerscj|}|jjdt||j||j y)Nr)dumpswritepackr/flush)rrdatarrs r_sendz$Autotuner._precompile.._sendsA<<$D LLT3t95 6 LL  LLNr!tensor_epilogue_fntensor_epilogue_sourcer#QUACK_COMPILE_WORKER_PARENT_PIDz-mz#torch._vendor.quack._compile_worker)rrstderrenvREADY) fn_module fn_qualnameargsr config_kwargszworker died before dispatchzquack-precompile-reader-T)rlrrdaemon)6r?rrqminr/rr*r7rhr all_kwargsr Exceptionr8r9printrrrrSrTr,rrRr+copyr4rgetpidrirPopensys executablePIPEDEVNULLrrrrrvrrrrrEventrsrrtThreadrstartrw) rPrrrr max_workerst_checkcurrentverboserrserialized_argsrr epilogue_fnepilogue_source worker_kwargs worker_envr)workersrnpreadyr assignmentsiconfigwirrrrs @@r _precompilezAutotuner._precompile~s. )$& &#g,BII6Ms,S(TU ! $& &))+ 6=WQZ%:%:%<=G DGGT %W % 99; 3 &$& &))|11344EFMQTT  N3w<.{m8T U  CGG31#6GGGG&& gg** jj!56  **%=>.tF|<  &;+B2M$$o3M. /ZZ__& 13  %1?J- .8;BIIK8H 45{# A  'LM oo!18z))d A&ahh/E NN1  $& &#$!4;'.rotating_callBsQ#Cj3/?+@@c a -c205+r!rCT) enable_timing) benchmarkerall)benchmark_itersmemory_warmup_iters return_modeis_vetted_benchmarking)r g?g?_AUTOTUNE_WARMUP_MS200_AUTOTUNE_TIMED_CALLS)rJwarmup_target_ms n_timed_calls quantileszAutotuning failed with : infc(jj jjijj dyy#t$r+} jj |#xYwd}~wwxYw)N)r)rr__call__rr)rrr- full_nargsrPs r kernel_callz%Autotuner._bench..kernel_calls}}( j)    ~~)zT:* ~~1zQ?E  s)A B'BB B  B  B)r[).r*r+r,r8r9r rrRrrrrArrnargsgetattrrrrr7rrir%r&rg CUDAGraphgraphreplayr(record elapsed_time$torch._inductor.runtime.benchmarkingrP benchmark_gpufloatmediantensoritemrrrKeyboardInterrupt SystemExitr typer)rPr9rmetar. conflicts has_hooks use_l2_cold bench_mode num_iters warmup_itersrNrnrer*endmsrP samples_msrYrZrrar-rJr`rKrLrMs` ` @@@@@@r_benchzAutotuner._benchs **..L$6$6$8#99J!KTRVYY  &tww'7'7&8 fXN OIIK&--"4"4"66 / )0D/EFJJ  t3v0023. .g. #4)%>),50<(-37 &?& #5<< Z0H#I#N#N#PQ(781B88#(II!3!3!5 66IJER$ !$II!3!3!5 66KLeT! 3GG$&!'!2!2!4%5"/-4 ;& >===H He,,89" B a"3Z!@A;A>3DG4D4D3ERsKLe eElE%LAA BL >/s34%L%,e = = >sj8D-S$%S-F+S$ S!S$$BS$?U#S S$$ U -A(UU U # V&,/V!V&!V&c |s|y|j}|Dcgc] }t|}}t|tt|k(sJdtt|g|z}t j dj|jdj}tt|}|jddd} |j| } | rtjj!t"j%ddst'||D cic]\} }| | } } }t)| d5} t+j,| d }|Dcic] \}}| || }}}t/j0||j |j2|<||_d |_dddy||j9t+j:||j4j=Dcgc]\}}t||fc}}d | d ycc}wcc}} wcc}}w#1swYyxYwcc}}w)NzConfig strings must be unique-rWz.autotune.json_FORCE_CACHE_UPDATEFrconfigs_timingsr_r)rMr)binary)rrr/rVERSIONrrrAencoderrDr`rRget_filer*r+r,r8r9zipopenjsonloadbuiltinsrr?r bench_timeputr r)rP tuning_keyrbench_fnrcconfig_str_list cache_keyr? file_namer@r str2configcached_configstimingsr9timings rcheck_disk_cachezAutotuner.check_disk_caches J  WW+23a3q633?#s3+?'@@aBaa@c*o.@ NN388I#6#=#=g#FGQQS  !34{{4C()8 ~~i(  ,*<*<*>)??R'SUZ[+.+HI41a!Q$IJIdC $N))N34EFLST.&&:f-v5TT)1g7;;)O :&'.$"#  $   JJ%FJFZFZF`F`Fb(3B67Vg.(    '4JU $ (s/H"( H'H3"H-2?H39H?-H33H<c @ttj_d}t j dkDrj ijjvr`d}jtjjfd}jrj|n|j}nj d}|_tj t"j%dddk(rF|sDt't"dj(j*d j,d d jd j(j.i|j1}d_|S) NTrFcjdi}tj}tjtj dddk(}tjtj dddk(}j duxsjdu}i}i}i}i} tj.|s, t} t| \} } | _ | _ nd_ d_ t!D]c\} } tj}|j#| tj|z || <|j%| r||rt'd| d|j(| dtj} t+fi| j-}j.i|tj|z || <tj}j2d| i|| <tj|z || <|st'd | d |j5| d d d |j5| d d d|| d d|| ddd f d_ d_ |j7tj}|r.|j9D]\} }t'd| d|dd d|r|rt'dt;|dt=|j?d dt=|j?d dt=|j?d dtj|z d d t=d|j?D}|rYt'd|dt;|dj.j@dtj d tBjD!||z _#tIjJ||j4"jL<|_'y#ttf$rd_ d_ Y6wxYw#t0$rYwxYw#d_ d_ |j7wxYw)#Nrr r_AUTOTUNE_WALL_DEBUGz[autotune] config z subprocess compile failed (z9); falling back to in-process compile before benchmarkingr9z[autotune-wall] config z: wait=gz.3fzs warm=zs bench=z s median=rz.6frz[z] -> z [autotune-wall-summary] configs=z wait_total=z s warm_total=zs bench_total=z s wall_total=rc3DK|]}|dtdk(sdyw)rr]rN)rk)rrs rrz8Autotuner.__call__..benchmark..Is "W!A$%PU,BV1"Ws  zquack autotune: /z configs failed for z; set z_PRINT_AUTOTUNING=1 for details)filer_)(r;rhr*r7r8r9rrrorrrr>r?rrrrrr rtrrrrr|r,rrr/sumvaluesrRr$rrrrr?r)r bench_startr. wall_debugrtr wait_times warm_timesbench_wall_times n_buffersarg_sets kwarg_setsr8r9 wait_start warm_startr-bench_wall_start bench_endtime_n_failedrrMrpruned_configsrPs r benchmarkz%Autotuner.__call__..benchmarks.T--tV^VvVF"&))+K ii<+=+=+?*@@Q(RTXY]``G!#l.@.@.B-CCW+XZ^!_cf!fJ $ T 9 WT^^SW=WI G!#J!#J')$I*#  >>1) @,A$,O 7N$(&)8" 4*:B 6;E 86:D27;D4)2>)B)"IAv *.J"OOA.,0IIK*,DJqM &//2#*$)*)?%& .2YY[ !).26.QV=N=N=P.QG$+DGGT$=W$= 15 j0H 1 /3yy{,.9dkk4.Y.YRX.YGFO26))+@P2P,Q/) %&=aSA,,6NN1c,B3+GH,,6NN1c,B3+GH--=a-@,EF..5foa.@-ER %I!"G)"Z26.370) $ I-4]]_EMFE!AfXU58C."CDE!g''*7|n5**-j.?.?.A*B3)GH**-j.?.?.A*B3)GH++./?/F/F/H+I#*NO**.))+ *CC)H K #"Wgnn.>"WWH.xj#g,H**.''*:*:);C5A##/#5#5#7"88WY"%  '0+&=DO&.ll7 &LDJJsO+2D(i%1+#>@:> 6;? 8 @F(1!)%)!)&26.370)s]-Q *P0B&Q ,P9A!Q &A Q P62Q 5P66Q 9 QQ QQ Q)rr rz autotuning for function z finished after z.2fzs; best config selected: ;)rrrrbr/rrr? prune_configsr%compilerdisablerr best_configr*r7r8r9r rrRrr_r) rPrrused_cached_resultrr9retrMrs ``` @@rr_zAutotuner.__call__s#dnnd34 ! t|| q &&'?$**'?'?@C$**$%*"!%!3!3F!;''z3(z3x%%))#~yIKZZ_F\\!_F! II++-..?@$ G3 N& . 9$'':J:J9KK[??3''@AQAQ@RRSU dgg   !    r!c *|j}|jr(|j|j|jfi|}tjt j d}|r tdt|}|d|}|jr|j}t|tr'|dkr"tt|j|z}nt|ts tdt||kDr^|Dcic]1}||jdi|j||j!3c}t#j%fdd|}|S#t$r#tt j d|wxYwcc}w) N_AUTOTUNE_MAX_CONFIGSrz._AUTOTUNE_MAX_CONFIGS must be an integer, got rzPError while pruning configs, top_k must be either 1) a float <= 1.0 or 2) an intc|Srr;)x est_timings rz)Autotuner.prune_configs..s TUr!r_r;)rrrbr*r7r8r9maxrrrrrrkr/ TypeErrorrrr)rPrrmax_configs_env max_configsrr9rs @rrzAutotuner.prune_configsns  " "4T44T\\4::XQWXN))|'9'9';&<"U*#1   ODOO** !++- "( (9?V!WX^Y^!_5  #))+,-*-/ "s/E!6F!,F )NNNF)rzOptional[Dict])rrq)rr rru)rRrSrTrK staticmethodrrrrr;r|r%rrrr_rr;r!rrrs +/?" ) ?"B* QfU>n ^^$ $ L]~"r!rc4eZdZdZdZdZdZdZdZdZ y) rz An object that represents a possible kernel configuration for the auto-tuner to try. :ivar kwargs: a dictionary of meta-parameters to pass to the kernel as keyword arguments. :type kwargs: dict[Str, Any] c ||_yrr)rPrs rrKzAutotuneConfig.__init__s  r!c2|jdi|_y)Nr)r,r)rPrMs r __setstate__zAutotuneConfig.__setstate__sii"- r!c|jSrrrs rrzAutotuneConfig.all_kwargss {{r!cg}|jjD]\}}|j|d|dj|S)Nr\r=)rrrrA)rPresrrs r__str__zAutotuneConfig.__str__sKKK%%' $DAq JJ!Bqc{ # $yy~r!cbtt|jjSr)hashrrrrs r__hash__zAutotuneConfig.__hash__s"E$//+113455r!ct|jj}t|jj}||k(Sr)rrr)rPother self_tuple other_tuples r__eq__zAutotuneConfig.__eq__sC4??,2245 E,,.4467 [((r!N) rRrSrTrrKrrrrrr;r!rrrs%. 6)r!rc^dtjdgfd}|S)Nzw Decorator for auto-tuning a function function. .. highlight:: python If the environment variable :code:`a_PRINT_AUTOTUNING` is set to :code:`"1"`, we will print a message to stdout after autotuning each kernel, including the time spent autotuning and the best configuration. :param configs: a list of :code:`AutotuneConfig` objects :type configs: list[AutotuneConfig] :param key: a list of argument names whose change in value will trigger the evaluation of all provided configs. :type key: list[str] :param prune_configs_by: a dict of functions that are used to prune configs, fields: 'perf_model': performance model used to predicate running time with different configs, returns running time 'top_k': number of configs to bench 'early_config_prune'(optional): a function used to do early prune (eg, num_stages). It takes configs:List[Config] as its input, and returns pruned configs. :param restore_value: a list of argument names whose value will be restored after evaluating any configs. :type restore_value: list[str] :param do_bench: a benchmark function to measure the time of each run. :type do_bench: lambda fn, quantiles :param cache_results: whether to cache autotune timings to disk. Defaults to False. "type cache_results: bool c (t|S)N)rrrr)r)rrrrrMrrs r decoratorzautotune..decorators%   '-'  r!)r8r9)rrMrrrrrs`````` rautotunersA ( )5(:(:(<'=> 2 {    r!)rrrr)rz str | None))NNNNT)7 __future__rrr*rr$rrhrrYrr dataclassesrrpathlibr functoolsrrtypingr r r r r bench.bench_utilsrrrr%rrrGr_compile_payloadrrr8rr r4r<rBruntimer?rDr`rorqrrrrrr;r!rrs#     (.33   Q$   .BE Gv~~++<< GL !V 9 9  9 x,<%Pz z z))B`d*r!