^jdZddlmZddlmZddlmZmZeje Z edeGdd eZ edeGd d eZ edeGd d eZ gdZy)zMllama model configuration)strict)PreTrainedConfig)auto_docstringloggingzmeta-llama/Llama-3.2-11B-Vision) checkpointcxeZdZUdZdZdZddiZdZee d<dZ e e d <d Z ee d <d Z ee d <dZee d<dZee d<dZee d<dZee d<dZeeezeeefze d<dZeeezeeefze d<dZee d<dZee d<dZeedze d<dZeeedze d<d Zee d!<fd"Zd#Zed$efd%ZxZ S)&MllamaVisionConfiga num_global_layers (`int`, *optional*, defaults to 8): Number of global layers in the Transformer encoder. Vision model has a second transformer encoder, called global. vision_output_dim (`int`, *optional*, defaults to 7680): Dimensionality of the vision model output. Includes output of transformer encoder with intermediate layers and global transformer encoder. max_num_tiles (`int`, *optional*, defaults to 4): Maximum number of tiles for image splitting. intermediate_layers_indices (`list[int]`, *optional*, defaults to [3, 7, 15, 23, 30]): Indices of intermediate layers of transformer encoder from which to extract and output features. These output features are concatenated with final hidden state of transformer encoder. supported_aspect_ratios (`list[list[int]]`, *optional*): List of supported aspect ratios for image splitting. If not specified, the default supported aspect ratios are [[1, 1], [1, 2], [1, 3], [1, 4], [2, 1], [2, 2], [3, 1], [4, 1]] for `max_num_tiles=4`. Example: ```python >>> from transformers import MllamaVisionConfig, MllamaVisionModel >>> # Initializing a Llama config >>> config = MllamaVisionConfig() >>> # Initializing a vision model from the mllama-11b style configuration >>> model = MllamaVisionModel(config) >>> # Accessing the model configuration >>> configuration = model.config ```mllama_vision_model vision_confignum_attention_headsattention_headsi hidden_sizegelu hidden_act num_hidden_layersnum_global_layersr num_channelsiintermediate_sizeivision_output_dimi image_size patch_sizeh㈵>norm_eps max_num_tilesNintermediate_layers_indicessupported_aspect_ratios{Gz?initializer_rangec |jddgddgddgddgddgddgddgddgg|_|j gd|_t| di|y)Nrr)r)r"r!super __post_init__selfkwargs __class__s z/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/mllama/configuration_mllama.pyr.z MllamaVisionConfig.__post_init__Msv  ' ' /-.FQFQFQFQPQFUVXYTZ]^`a\befhidj+kD (  + + 3/AD , ''c |jddgddgddgddgddgddgddgddggk(r|jdk7r tdyy)zOPart of `@strict`-powered validation. Validates the architecture of the config.r&r'rrz;max_num_tiles must be 4 for default supported aspect ratiosN)r"r ValueErrorr0s r3validate_architecturez(MllamaVisionConfig.validate_architectureUsr  ( (aVaVaVaVaQRVVWYZU[^_ab]cfgijek,l l""a'Z[ [( mr4returnc,t|jS)N)lenr"r7s r3max_aspect_ratio_idz&MllamaVisionConfig.max_aspect_ratio_id]s4//00r4)!__name__ __module__ __qualname____doc__ model_typebase_config_key attribute_maprint__annotations__rstrrrrrrrrlisttuplerrfloatr r!r"r$r.r8propertyr< __classcell__r2s@r3r r s" <'J%O*,=>MKJssOSL#!s!!s!47Jd3i%S/1746Jd3i%S/16HeM348cT!186:T$s)_t3:#u#(\1S11r4r c\eZdZUdZdZdZdZdZee d<dZ ee d<d Z e e d <d Z ee d <d Zee d<dZee d<dZee d<dZedze d<dZee d<dZee d<dZee d<dZee d<dZee d<dZeedze d<d Zeeze d!<d"Zee d#<d$Zeeezdze d%<d&Zedze d'<fd(Z xZ!S))MllamaTextConfiga cross_attention_layers (`list[int]`, *optional*): Indices of the cross attention layers. If not specified, will default to [3, 8, 13, 18, 23, 28, 33, 38]. Example: ```python >>> from transformers import MllamaTextModel, MllamaTextConfig >>> # Initializing a Mllama text config >>> config = MllamaTextConfig() >>> # Initializing a model from the Mllama text configuration >>> model = MllamaTextModel(config) >>> # Accessing the model configuration >>> configuration = model.config ```mllama_text_model text_configgA vocab_sizeirsilur(rrr rnum_key_value_headsi8rNrope_parametersr rms_norm_epsimax_position_embeddingsr#r$T use_cacheFtie_word_embeddingscross_attention_layersgdropouti bos_token_idi eos_token_idi pad_token_idc N|j gd|_t|di|y)N)rr r*!&r,)r[r-r.r/s r3r.zMllamaTextConfig.__post_init__s'  & & .*HD ' ''r4)"r=r>r?r@rArB default_thetarRrDrErrrFrr rUrrVdictrWrIrXr$rYboolrZr[rGr\r]r^r_r.rKrLs@r3rNrNbs &%J#OMJKJs!!  #s##'OTD['L%#*S*#u#It %%/3DI,3GUS[L#+1L#S /D(1%L#*%((r4rNc|eZdZUdZdZddiZeedZdZ e e zdze d<dZ e e zdze d<d Zee d<fd ZxZS) MllamaConfiga Example: ```python >>> from transformers import MllamaForConditionalGeneration, MllamaConfig, MllamaVisionConfig, MllamaTextConfig >>> # Initializing a CLIP-vision config >>> vision_config = MllamaVisionConfig() >>> # Initializing a Llama config >>> text_config = MllamaTextConfig() >>> # Initializing a mllama-11b style configuration >>> configuration = MllamaConfig(vision_config, text_config) >>> # Initializing a model from the mllama-11b style configuration >>> model = MllamaForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```mllamaimage_token_idimage_token_index)rPr Nr rPrQc |j%t|_tjdn4t |jt rtdi|j|_|j %t|_tjdn4t |j t rtdi|j |_t|$di|y)Nz9vision_config is None, using default mllama vision configz5text_config is None, using default mllama text configr,) r r loggerinfo isinstancergrPrNr-r.r/s r3r.zMllamaConfig.__post_init__s    %!3!5D  KKS T **D 1!3!Id6H6H!ID     #/1D  KKO P (($ //C$2B2BCD  ''r4)r=r>r?r@rArCrNr sub_configsr rgrrErPrmrDr.rKrLs@r3rjrjsh ,J-M#3EWXK48M4**T1826K((4/6#s# ( (r4rj)rjrNr N)r@huggingface_hub.dataclassesrconfiguration_utilsrutilsrr get_loggerr=ror rNrj__all__r,r4r3rxs!.3,   H %<=E1)E1>E1P<=.('.(>.(b<=.(#.(>.(b Er4