^j#dZddlZddlZddlmZdgZej eZde fdZ edZ e jdZ e jd Zy) aQCompile-safe backward gradient logging for multiple tensors. ``debug_grad_log`` logs gradient norms during backward for one or more tensors. It is a leaf function with a ``register_multi_grad_hook`` that fires exactly once when all requires_grad tensor inputs have their gradients computed. Example:: import torch from torch.utils.debug_log import debug_grad_log x = torch.randn(4, requires_grad=True) y = torch.randn(4, requires_grad=True) z = x * 2 + y * 3 debug_grad_log(x, y) z.sum().backward() # Logs: [rank 0][bwd] t0_grad_norm=... t1_grad_norm=... N) leaf_functiondebug_grad_logreturnctjjsyddlm}|j r|j SdS)Nr)torch distributed is_availabletorch.distributedis_initializedget_rank)dists `/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/torch/utils/debug_log.py _get_rankr!s6    ) ) +$"1134==?::cy)aLog gradient norms of multiple tensors during backward. This is a no-op in the forward pass. During backward, the hook fires exactly once when all requires_grad tensor inputs have their gradients computed, and logs ``[rank R][bwd] t0_grad_norm=... t1_grad_norm=...``. Args: *tensors: One or more tensors to monitor. Returns: None. Call without assignment: ``debug_grad_log(x, y)``. Ntensorss rrr)s rcy)Nrrs r_debug_grad_log_faker:s rcdjdt|D}tjdt |y)N c3lK|],\}}d|d|jjd.yw)tz _grad_norm=z.4fN)normitem).0igs r z'_debug_grad_log_hook..As548Aq!A3k!&&(--/#./s24z[rank %d][bwd] %s)join enumerateloginfor)gradsnormss r_debug_grad_log_hookr'?s7 HHr1s* 2  g!;3;   ((6)6r