^j&ddlmZddlmZddlmZddlmZmZddl m Z eje Z ed eGd d eZed eGd d eZd d gZy))Any)strict)PreTrainedConfig)auto_docstringlogging)SiglipVisionConfigzgoogle/gemma-3-4b-it) checkpointc 4eZdZUdZdZdgZdddddddddd Zdgd gfd d gd gfd gd gfd Zd Ze e d<dZ e e d<dZ e e d<dZ e e d<dZe e d<dZe e d<dZe e d<dZee d<dZe e d<dZee d <d!Zee d"<d#Zee d$<d%Ze d&ze d'<d(Ze ee zd&ze d)<d*Ze d&ze d+<d#Zee d,<d&Zed&ze d-<d.Z ee d/<d0Z!e ezd&ze d1<dZ"e e d2<d3Z#e d&ze d4<d&Z$eed&ze d5<d&Z%ed&ze d6<d&Z&ed&ze d7<d.Z'ed&ze d8<d9d:d;Z(fd<Z)d=Z*d>Z+xZ,S)?Gemma3TextConfiga query_pre_attn_scalar (`float`, *optional*, defaults to 256): scaling factor used on the attention scores final_logit_softcapping (`float`, *optional*): Scaling factor when applying tanh softcapping on the logits. attn_logit_softcapping (`float`, *optional*): Scaling factor when applying tanh softcapping on the attention scores. use_bidirectional_attention (`bool`, *optional*, defaults to `False`): If True, the model will attend to all text tokens instead of using a causal mask. This does not change behavior for vision tokens. ```python >>> from transformers import Gemma3TextModel, Gemma3TextConfig >>> # Initializing a Gemma3Text gemma3_text-7b style configuration >>> configuration = Gemma3TextConfig() >>> # Initializing a model from the gemma3_text-7b style configuration >>> model = Gemma3TextModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` gemma3_textpast_key_valuescolwisereplicated_with_grad_allreducerowwise) zlayers.*.self_attn.q_projzlayers.*.self_attn.k_projzlayers.*.self_attn.v_projzlayers.*.self_attn.q_normzlayers.*.self_attn.k_normzlayers.*.self_attn.o_projzlayers.*.mlp.gate_projzlayers.*.mlp.up_projzlayers.*.mlp.down_proj input_ids inputs_embeds hidden_statesattention_mask) embed_tokenslayersnormi@ vocab_sizei hidden_sizei$intermediate_sizenum_hidden_layersnum_attention_headsnum_key_value_headshead_dimgelu_pytorch_tanhhidden_activationimax_position_embeddings{Gz?initializer_rangegư> rms_norm_epsT use_cacherN pad_token_id eos_token_idr bos_token_idtie_word_embeddingsrope_parametersFattention_biasgattention_dropoutquery_pre_attn_scalarisliding_window layer_typesfinal_logit_softcappingattn_logit_softcappinguse_bidirectional_attentiong.Ag@)globallocalc D|jr|jdzdz|_|jdd|_|jEt |j Dcgc]!}t|dz|jzrdnd#c}|_t|$di|ycc}w)Nr r-sliding_window_patternsliding_attentionfull_attention) r9r5get_sliding_window_patternr6rangerboolsuper __post_init__)selfkwargsi __class__s z/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/gemma3/configuration_gemma3.pyrGzGemma3TextConfig.__post_init__hs  + +#'#6#6!#;q"@D (.zz2JA'N$    #t556 (,QUd6R6R,R'S#Yii D  ''  s&Bc|j|jzdk7r&td|jd|jdy)zOPart of `@strict`-powered validation. Validates the architecture of the config.rzThe hidden size (z6) is not a multiple of the number of attention heads (z).N)rr ValueError)rHs rLvalidate_architecturez&Gemma3TextConfig.validate_architecturewsS   d66 6! ;#D$4$4#5622327  <c z|jdd}ddiddid}|j |jn||_||jdj||jjdddi|jd<|jdj d|jd|j d|jjdddi|jd<|jdj d|jd |j d |j |S) N rope_scaling rope_typedefault)r?r@r@ rope_thetar:r?rope_local_base_freqr;)popr1updaterB setdefault default_thetastandardize_rope_params)rHrIrRdefault_rope_paramss rLconvert_rope_params_to_dictz,Gemma3TextConfig.convert_rope_params_to_dictsIzz.$7 #.y!9*I6 8<7K7K7Wt33]p  #  !1 2 9 9, G    # #$4 5 =6A95MD !1 2 -.99 &**\43E3Eh3OP     # #$7 8 @9Di8PD !4 5 01<< &**%;T=O=OPW=XY $$& rP)-__name__ __module__ __qualname____doc__ model_typekeys_to_ignore_at_inferencebase_model_tp_planbase_model_pp_planrint__annotations__rrrr r"r$r&strr'r)floatr*r+rEr,r.listr/r0r1dictr2r3r4r5r6r7r8r9rZrGrOr] __classcell__rKs@rLr r !s,J#4"5%.%.%.%E%E%."+ )"+ &(9:#%568IJ!"_$56 JK!s!s    Hc0s0#*S*#u#L%It L#* +,L#S /D(, L#* $$#'OTD[' ND ,/sU{T)/!$3$!%NC$J%$(KcT!(,0UT\0+/EDL//44*X>M (rPr ceZdZUdZdZddddZeedZdZ ee e e fzdze d <dZee e e fzdze d <d Zedze d <d Zedze d<dZedze d<dZedze d<dZedze d<dZedze d<fdZxZS) Gemma3Configa mm_tokens_per_image (`int`, *optional*, defaults to 256): The number of tokens per image embedding. boi_token_index (`int`, *optional*, defaults to 255999): The begin-of-image token index to wrap the image prompt. eoi_token_index (`int`, *optional*, defaults to 256000): The end-of-image token index to wrap the image prompt. Example: ```python >>> from transformers import Gemma3ForConditionalGeneration, Gemma3Config, SiglipVisionConfig, Gemma3TextConfig >>> # Initializing a Siglip-like vision config >>> vision_config = SiglipVisionConfig() >>> # Initializing a Gemma3 Text config >>> text_config = Gemma3TextConfig() >>> # Initializing a Gemma3 gemma-3-4b style configuration >>> configuration = Gemma3Config(vision_config, text_config) >>> # Initializing a model from the gemma-3-4b style configuration >>> model = Gemma3TextConfig(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```gemma3image_token_indexboi_token_indexeoi_token_index)image_token_id boi_token_id eoi_token_id) text_config vision_configNrwrxr#mm_tokens_per_imageiiir(r)Tr0c |j%t|_tjdn4t |jt rtdi|j|_t |j t rtdi|j |_n0|j $t|_tjdt|$di|y)Nz@text_config is None, using default Gemma3TextConfig text config.zFvision_config is None, using default SiglipVisionConfig vision config.rA) rwr loggerinfo isinstancerkrxr rFrG)rHrIrKs rLrGzGemma3Config.__post_init__s    #/1D  KKZ [ (($ //C$2B2BCD  d(($ /!3!Id6H6H!ID     '!3!5D  KK` a ''rP)r^r_r`rarb attribute_mapr r sub_configsrwrkrhrrgrxryrfrrrsrqr)rir0rErGrlrms@rLroros :J-))M (+K =AK!DcN2T9@@DM%S#X6=D&)t)")OS4Z)")OS4Z)$+sTz+&*ut|*'++ ( (rProN)typingrhuggingface_hub.dataclassesrconfiguration_utilsrutilsrrsiglipr get_loggerr^r{r ro__all__rArPrLrs*.3,'   H %12w'w3wt12?(#?(3?(D - .rP