^jFUddlmZddlZddlZddlZddlZddlZddlmZddlm Z m Z ddl m Z ddl Z ddlmZmZe rddlmZmZe dZdd a d dd Zdd Zdd Zdd ZddZddZgaded<ddZ d ddZy)) annotationsN)AbstractContextManager)Any TYPE_CHECKING)TypeVar)profileProfilerActivity)CallableSequence_Rcy)Nrk/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/_functorch/benchmark_utils.py synchronizersrc |dg}|dgk7r8tjjrtjja|i}|i}|5tjdt dD]} ||fi|ttjdt j} t |D]} ||fi|tt j} ddd  z } tdd|i|5} |5ttjdt |D]} ||fi|t dddddd j|| S#1swYxYw#1swY0xYw#1swY4xYw)a1 Output the chrome trace of running f(input_, **kwargs_for_f) with [optimize_ctx] [num_runs] times to [trace_filename]. [activities] are the activities that the profiler will record, e.g. ProfilerActivity.CUDA. Return total runtime without the profiler Outputs to trace_filename Ncudacpui9 activitiesr) torch accelerator is_availabler manual_seedrangetime perf_counterrexport_chrome_trace)finput_trace_filename optimize_ctxrnum_runsdevices kwargs_for_fkwargs_for_profiler_t0t1timingprofs rdump_chrome_tracer,s*(5'e//<<>''33  "   ! $q A f % % M  $    x A f % % M     !"WF  >J >*= >$   M   d #8_ &)L)    ^, M- ! !  s2BE+FAE7F+E47F <FF czt|5}tj|}dddd}|S#1swYxYw)N traceEvents)openjsonload)filenamerdataeventss rget_chrome_trace_eventsr5Us< h1yy| - F Ms1:cDd|vxr|dtvxrd|vxr|ddk(S)NpidphX)gpu_pidsevents ris_gpu_compute_eventr=\s@   %LH $  EM  $K3  rcg}|D]}t|s|j|!t|tjdS)Nts)key)r=appendsortedoperator itemgetter)r4sorted_gpu_eventsr<s rget_sorted_gpu_eventsrFfsK.0(#E*   '( #)<).is_mm_conv_event~sT eFm # (v& (E&M) (%-'  rr<zdict[str, Any]returnbool)rFrA)r4rX gpu_events sorted_eventsr<s rget_sorted_gpu_mm_conv_eventsr^}sH 'v.J*,M$& U#$ rz list[Any]r:ct|}ga|D]3}d|vr|ddk(sd|ddvstj|d5|dz}t|}t ||z }t |}t ||z }||fS)a Process the chrome traces outputs by the pytorch profiler to compute GPU Utilization and percent of times spent on matmul and convolution Args: filename(str): Name of chrome traces file produced by pytorch profiler total_length(float): total length of the process without profiler in second Return: tuple: (GPU Utilization, percent of time spent on matmul and convolution) rSprocess_labelsGPUargslabelsr7g.A)r5r:rArFrPr^)r2 total_lengthr4r<rE utilizationsorted_gpu_mm_conv_eventsmm_conv_utilizations rcompute_utilizationrhs%X .FH*    =, ,%-:Q1Q OOE%L ) *  #%L-f501L@K =f E&'@ALP + ++rc ftjj|}|s#tj|t d|z|t j }tjj||dz}t||||tjg|dg}t||\} } | | fS)a Benchmark the GPU Utilization and percent of time spent on matmul and convolution operations of running f(input_, **kwargs_for_f) with [optimize_ctx] [num_runs] times. It will produce a chrome trace file in trace_folder/trace_file_name.json Example: ``` def f(a): return a.sum() a = torch.rand(2**20, device="cuda") utilization, mm_conv_utilization = benchmark_utilization( f, a, "tmp", trace_file_name="tmp_chrome_trace" ) ``` Args: f: function to benchmark input_: input to :attr:`f` trace_folder: name of the folder to store the chrome trace optimize_ctx: the context in which f will run trace_file_name: name of the dumped chrome trace file, default to "tmp_chrome_trace" num_runs: number of times to run f, excluding the warm-up runs, default to 1. Return: tuple: (GPU Utilization, percent of time spent on matmul and convolution) zcreate folder z.jsonr)r#r$) ospathexistsmakedirsprint contextlib nullcontextjoinr,r CUDArh) rr trace_folderr"trace_file_namer#isExistchrome_trace_file_namerdrergs rbenchmark_utilizationrwsVggnn\*G  L! -.!--/ WW\\,'8QR$    L(; ($K$ + ++r)rZNone)rINNN)rCallable[[tuple[Any, ...]], _R]r tuple[Any, ...]r!strr"zAbstractContextManager[Any]rzSequence[ProfilerActivity]r#intr$zlist[str] | Noner%dict[str, Any] | Noner&r}rZfloat)r2r{rZlist[dict[str, Any]]rY)r4rrZr)rErrZr|)r2r{rdr~rZtuple[float, float])Ntmp_chrome_tracerI)rryr rzrsr{r"z"AbstractContextManager[Any] | Nonertr{r#r|rZr) __future__rror0rCrjrrtypingrrtyping_extensionsrrtorch.profilerrr collections.abcr r r rr,r5r=rFrPr^r:__annotations__rhrwrrrrs8"  -%% 42 T]  $*.157&7 77. 7 + 7  77(7/7 7tD $),L8<- A,&A, A,A,5 A,  A,  A,A,r