^jBLddlZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZddlmZddlmZdd lmZmZej.eZej4d Gd d Zej4d GddZej4d GddZej4d GddZej4d GddZej4d GddZ GddZ!y)N)Path)counters dynamo_timedset_feature_use)justknobs_check)FileLock) OrderedSet)triton_cache_dir) _IS_WINDOWSGPU_KERNEL_BIN_EXTST)frozenc0eZdZUdZeed<eed<eed<y)TritonBundleEntryz When we have compiled a triton kernel, we take note of that kernel by its triton generated hash, its device, and where this kernel is located. This is the minimum information we can use to later retrieve this kernel from file system. kernel_hashdevice directoryN)__name__ __module__ __qualname____doc__str__annotations__inti/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/_inductor/triton_bundler.pyrrs KNrrcLeZdZUdZeed<ejdZe ed<y)TritonKernelArtifactzq Artifact for an individual kernel converted to bytes. Bytes could be a cubin, json, ttir, or ttgir. filenameF)reprpayloadN) rrrrrr dataclassesfieldr"bytesrrrrr#s% M&[&&E2GU2rrc0eZdZUdZeed<eed<ded<y)StaticallyLaunchedAutotuneraQ Represents a statically compiled CachingAutotuner object that we can save directly in the cache. A CachingAutotuner is made up of a list of StaticTritonCompileResults, each of which uses the cubin from a TritonKernelArtifact. Statically saved here have their cubin files saved by a corresponding TritonBundleEntry. cache_key kernel_nameCachingAutotunerkernelN)rrrrrrrrrr'r'.sN rr'c6eZdZUdZeed<eed<eeed<y)TritonKernelArtifactsz: Collection of artifacts for a particular kernel. rr artifactsN) rrrrrrrlistrrrrr-r-=s  K())rr-c2eZdZUdZeeed<eeed<y)TritonBundlerMetadataz+ Metadata used for instrumentation cached_kernel_names statically_launched_kernel_namesN)rrrrr/rrrrrr1r1Hsc"&*3i/rr1c2eZdZUdZeeed<eeed<y) TritonBundlez7 Serializable bundle to save into FXGraphCache kernel_artifactsstatic_autotunersN)rrrrr/r-rr'rrrr5r5Rs!011788rr5ceZdZUdZdZeedzed<dZee dzed<dZ e e dzed<dZ eed<edefd Zedd Zedd Zed e d eddfdZed e ddfdZede ddddfdZedeee ee ffdZedee dzdee fdZedeeedzffdZedededzfdZy) TritonBundlera Lightweight Triton Kernel bundler that notes each time we compile a triton kernel. When collect is called, converts all the previously noted kernels and their artifacts into a structured bytes blob, and later when write is called it writes this structured blob back to file system. Intended Life cycle: - TritonBundler.begin_compile is called when we start compiling in Inductor - TritonBundler.put is called each time a Triton Kernel is compiled - TritonBundler.collect is called when a cache entry is being generated - TritonBundler.end_compile is called to indicate bundling is completed, collect will execute this function as well. - TritonBundler.read_and_emit is called when a cache entry is read N_entries_static_autotuners_winnerss [REPLACE]_REPLACE_BYTESreturncddlm}|jry|jx}|S|j syt dS)NrconfigFz9pytorch/remote_cache:bundle_triton_into_fx_graph_cache_v2)torch._inductorrAforce_disable_caches!bundle_triton_into_fx_graph_cache is_fbcoder)rAbs r is_enabledzTritonBundler.is_enabledtsF*  & &99 9A FH! G  rctjsytjd|jJg|_g|_t |_y)z Initializes the TritonBundler. The current TritonBundler bundle is finalized by TritonBundler.collect. Nz%TritonBundler.begin_compile is called)r9rGlogdebugr:r;r r<clss r begin_compilezTritonBundler.begin_compilesH '')  9:||### !#!| rcXtjdd|_d|_d|_y)zt Finalizes the TritonBundler. If collect is not yet called, it discards the current bundle. z#TritonBundler.end_compile is calledN)rIrJr:r;r<rKs r end_compilezTritonBundler.end_compiles' 78 !% rrrc l|jx}&|jt||t|yy)z Lazily observes that we have seen a Triton kernel compilation. Remembers it for when collect is later called. N)r:appendrr )rLrrentriess rputzTritonBundler.puts6 || #G 0 NN!+v7G7OP  1rcT|j|jj|yy)a Marks a kernel hash as a winning autotuning config. Only winning kernels are included in the bundle by collect(). If no winners are recorded (e.g. single-config kernels that skip autotuning), all entries are bundled. N)r<add)rLrs r put_winnerzTritonBundler.put_winners% << # LL  [ ) $rkeyr+r*c Pddlm}|jsJ|jx}|j }t j |}|jd|_|jt||jjdd||j|yy)Nrr@r)unknown_kernel)rBrAuse_static_triton_launcherr;prepare_for_picklecopydeepcopyprepare_for_caching_reload_kernelrQr' inductor_metagetrestore_after_unpickle)rLrWr+rArR old_values new_kernels rput_static_autotunerz"TritonBundler.put_static_autotuners*0000-- -G : 224Jv.J  * * ,(,J % NN+,,00@PQ   ) )* 5# ;rc|jsggfStjdt|j|jDcgc]}|j}}t ddxxdz cc<|j|fScc}w)Nz1Saving %d statically launchable CachingAutotunersinductor$triton_bundler_save_static_autotunerr )r;rIinfolenr)r)rListatic_autotuner_namess rcollect_static_autotunersz'TritonBundler.collect_static_autotunerss%%r6M HHCC**+ >A=S=S%Tamm%T "%T Z !G HA M H))+AA A&UsA?r7c&|sgSddlm}ddlm}tj dt |g}td5|D]} |jjD]}|j ||j|j|j<t d d xxd z cc<|j#|j d d d |S#t$r%tjd|jdYwxYw#1swY|SxYw) zv Load statically launchable CachingAutotuners into async_compile.CompiledTritonKernels cache. r)CompiledTritonKernels)StaticAutotunerFuturez+Loading %d statically launchable autotunersz+TritonBundler.load_cached_static_autotunersz>Failed to reload cubin file statically launchable autotuner %sTexc_inforg$triton_bundler_load_static_autotunerr N)torch._inductor.async_compilerotorch._inductor.codecacherprIrirjrr+compile_resultsreload_cubin_path RuntimeErrorwarningr)_cacher(rrQ)rLr7rorp kernel_namesresultcompile_results rload_autotunerszTritonBundler.load_autotunerss!IGC >DU@VW G H 8+ 8 *0--*G*G;&88:;BWMMB%,,V-=-=>$%KLPQQL##F$6$67) 8 8,!$KKX**!%     8,s0D+C/AD+DDDDDc ddlm}tjs*|j t ddt ggdfSt ddtdd5|j}||j}g}g}|D]T}|r/|j|vr!tjd |j5g}tjj|j |j}tjj#|stj$|D]}} tjj|| } tjj'| sJt)| d 5} | j+} | j-d rutj.| vr0tj1d tj.| t3d | j5t6j9|tj.} |j;t=| | dddt>ddxxdz cc<tjjC| d} | tEjFvsZ|j;tI| jJ|s%|j;tM|j|jN|W|jPr|jS\}}ng}g}|j t ||tU||fcdddSt ggdfcdddS#1swY"xYw#t@$rtjddY4wxYw#1swYyxYw)a This is the main function called when a cache write happens. This function converts all the previously remembered kernels into bundled format so that it can be written into a cache entry. This function also finalizes the current bundle. rr@triton_bundlingFNTzTritonBundler.collectrWlog_pt2_compile_eventzSkipping non-winning kernel %srb.jsonz'Bundle contains illegal %s, payload: %szBundle contains illegal bytesrgtriton_bundler_save_kernelr zfailed to collect triton kernelrq)+rBrAr9rGrOrr5rr:r<rrIrJospathjoinrexistslistdirisfileopenreadendswithr=ryAssertionErrorreplacerencoderQrr Exceptionsplitextr valuesrstemr-rrZrmr1)rLrArRwinnersr|r{entryr.rr filepathfiler" extensionr7static_kernel_namess rcollectzTritonBundler.collects* +'') OO  -u 5B'- -)40 5T RE .llG" ,,68*, $/E5#4#4G#C "BEDUDUV <>I77<<9J9JKD77>>$/ $&JJt$4E#%77<<h#?X#%77>>(#;;#;!%h!5"*.))+#+#4#4W#='4'C'Cw'N(+ ,U,9,H,H,3)* /=,K/*)*/6oo(+ 4(8-:V:V/&G!* 0 0$87$K!"#"(%Z01MNRSSN%'GG$4$4X$>q$A $(;(B(B(DD)//X0C0CD?E@! 1 % 1 1 % )S/`44557;%':)+%*,'!#F,=>@U "5AEE .E .J B'-KE .E .,""* )XII&GRVIWXWE .E .s]C&M8-M2B2M $M8M8:)M8%A>M8- M8M M M5 1M84M5 5M88Nbundlec 0ddlm}tjsyt dd5g}|j D]}t |j}tjj||j}tjj|r8ttj|dk7rtj!d|t#|j%ddt't)j*}tjj|d |}tj,||j.D]3}tjj||j0} t3| d 5} |j4} |j0j7d r3| j9tj:t&j=|} | j?| dddt@d d xxdz cc<tjjC|j0d} | tEjFvs|jIt#|j0jJ6tLrctO|dz5tjj|rtQjR|tj8||ddd tj8|||jXr tj[|j\} ng} t_|| cdddS#1swYYxYw#1swY+xYw#tT$rtjWd|YPwxYw#1swYyxYw)a This is the main function called when a cache read happens. This function converts the bundled format back into individual files and writes them to the filesystem. NOTE: When we are writing to the filesystem, we assume exclusive access to the target directory. This means that if the target folder already exists and is non-empty, we bail out. Exclusive access means that no other process should be writing to or reading from the target directory. rr@NzTritonBundler.read_and_emitTrz8Bailing out TritonBundler.read_and_emit, %s is non empty)parentsexist_okztmp.wbrrg#triton_bundler_read_and_emit_kernelr z.lockz%Directory %s is not empty - skipping!)0rBrAr9rGrr6r rrrrrrrjrrIrJrmkdirruuiduuid4makedirsr.r rr"rrr=rwriterrr rrQrr rshutilrmtreeOSErrorryrZr~r7r1)rrAr{r.basedirrrnd_idtmp_dirartifactrrr"rrs r read_and_emitzTritonBundler.read_and_emit\s +'') -T ; L')L#44. V *9+;+;<GGLL)2G2GH 77>>),RZZ 5J1Kq1PIIR!W ##D4#@TZZ\*'',,w$vh@ G$ ) 3 3JH!ww||GX5F5FGHh-,"*"2"2#,,55g>&-oo - < S'G 7+ ,Z()NOSTTO " 0 01B1B CA FI $7$>$>$@@%++D1B1B,C,H,HIJ !)g"56777>>)4"MM)4 7I677 V 7I6Y. V`00&3&C&C,,'#')#(7JKw; L; L6,,77#V $KWUVe; L; LsjE N  A,M 5AN AN A M% N 0M&N) rrrrr:r/rrr;r'r<r rr=r% staticmethodboolrG classmethodrMrOrrSrVretuplermr~r5r1rrrrrr9r9\s 04Hd$%,3CG89D@G'+Hjo$+)NE(    $ $c34*S*T**6s64F6466. B t/0$s); < B B% $%@ AD H% c%%NV. |2T99 :V.V.pMLlML/Dt/KMLMLrr9)"r\r#loggingrrrpathlibrtorch._dynamo.utilsrrrtorch._utils_internalrtorch.utils._filelockrtorch.utils._ordered_setr runtime.runtime_utilsr utilsr r getLoggerrrI dataclassrrr'r-r1r5r9rrrrs5  GG1*/33g!d#  $ d#33$3d#  $ d#**$*d#00$0d#99$9NLNLr