^j<dZddlZddlZddlmZddlddlmZmZddl m Z m Z m Z m Z mZmZmZmZmZmZmZddlmZdd lmZed,id e d e d e d e de dedededededededededede de!dededededed ed!ed"eZ"e"jGDchc]\}}| c}}Z$ee e e e e eeeeeee# Z%e e e e eeeeeeeeeh Z& d-d$ed%e'd&eed'e(d(e(f d)Z) d.d*ed&eefd+Z*ycc}}w)/aU NormAct (Normalization + Activation Layer) Factory Create norm + act combo modules that attempt to be backwards compatible with separate norm + act instances in models. Where these are used it will be possible to swap separate BN + act layers with combined modules like IABN or EvoNorms. Hacked together by / Copyright 2020 Ross Wightman N)Optional)*)FilterResponseNormAct2dFilterResponseNormTlu2d) BatchNormAct2d GroupNormAct GroupNorm1Act LayerNormActLayerNormActFp32LayerNormAct2dLayerNormAct2dFp32 RmsNormActRmsNormActFp32 RmsNormAct2dRmsNormAct2dFp32) InplaceAbn) LayerType batchnorm batchnorm2d groupnorm groupnorm1 layernorm layernorm2d layernormfp32layernorm2dfp32 evonormb0 evonormb1 evonormb2 evonorms0 evonorms0a evonorms1 evonorms1a evonorms2 evonorms2afrnfrntlu inplaceabniabnrmsnorm rmsnorm2d rmsnormfp32 rmsnorm2dfp32) rrrrrrrrr*r+r,r- layer_name num_features act_layer apply_actjitc xt||}||fd|i|}|rtjj|}|S)N)r0r1)get_norm_act_layertorchr2script)r.r/r0r1r2kwargslayerlayer_instances f/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/timm/layers/create_norm_act.pycreate_norm_act_layerr;\sA zY ?E<G9GGN )).9  norm_layerc8|yt|tttjt j fsJ|6t|tttjt j fsJi}t|t j r'|j|j|j}t|tr?|sy|jddjjdd}t|}nd|tvr|}nYt|tjr|}n<|jj}t j#|d}| Jd||t$vr|j'd||rt j |fi|}|S)N_-rz!No equivalent norm_act layer for r0) isinstancetypestrtypes FunctionType functoolspartialupdatekeywordsfuncreplacelowersplit _NORM_ACT_MAP_NORM_ACT_TYPES__name___NORM_TO_NORM_ACT_MAPget_NORM_ACT_REQUIRES_ARG setdefault)r=r0norm_act_kwargsr.norm_act_layer type_names r:r4r4ksr j4u/A/A9CTCT"U VV V   9tS%BTBTV_VgVg6h ii iO*i//0z223__ *c"''R0668>>sCAF &z2  &# J!3!3 4#''--/ .229dC)Z-Nyk+ZZ)// ""; :"**>M_M r<)NTF)N)+__doc__rErGtypingrevo_normfilter_response_normrrnorm_actrr r r r r rrrrr inplace_abnrrdict EvoNorm2dB0 EvoNorm2dB1 EvoNorm2dB2 EvoNorm2dS0 EvoNorm2dS0a EvoNorm2dS1 EvoNorm2dS1a EvoNorm2dS2 EvoNorm2dS2arOitemsrPrRrTintboolr;r4)nms00r:ros R    $     #' !"#$ %& #'()* +, -./012#3 6"/!4!4!67A17"& "  (*.   I&    "*.&&I&&c8s( C>