^j<dZddlmZddlmZddlmZddlmZm Z e je Z edeGd d eZ edeGd d eZd gZy )zEvolla model configuration)strict)PreTrainedConfig)RopeParameters)auto_docstringloggingzwestlake-repl/Evolla-10B-hf) checkpointceZdZUdZdZeed<dZeed<dZeed<dZ eed <d Z eed <d Z eed <dZ eed<dZ eezed<dZeezed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed <dZeed!<dZeed"<y#)$ SaProtConfiga mask_token_id (`int`, *optional*, defaults to 4): The id of the *mask* token in the protein sequence model. position_embedding_type (`str`, *optional*, defaults to `"rotary"`): The type of position embedding to use in the protein sequence model. Currently only `"rotary"` is supported. rope_theta (`float`, *optional*, defaults to 10000.0): The base period of the RoPE embeddings. emb_layer_norm_before (`bool`, *optional*, defaults to `False`): Whether to apply layer normalization before the position embedding in the protein sequence model. token_dropout (`bool`, *optional*, defaults to `True`): Whether to apply dropout to the tokens in the protein sequence model. i vocab_size mask_token_id pad_token_idi hidden_size!num_hidden_layersnum_attention_headsiintermediate_size皙?hidden_dropout_probattention_probs_dropout_probimax_position_embeddings{Gz?initializer_rangeh㈵>layer_norm_epsrotaryposition_embedding_typeg@ rope_thetaFemb_layer_norm_beforeT token_dropout is_decoderadd_cross_attentionN)__name__ __module__ __qualname____doc__r int__annotations__rrrrrrrfloatrrrrr strr!r"boolr#r$r%z/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/evolla/configuration_evolla.pyr r s JM3L#Ks!!!s!'**03 %#+3#'S'#u#!NE!#+S+J"'4'M4J %%r0r cpeZdZUdZdZdeiZdZdZe e zdze d<dZ e e d<dZe e d <d Ze e d <d Ze e d <d Ze e d<dZe dze d<dZee d<dZe e d<dZee d<dZee zdze d<dZee d<dZee zdze d<dZee d<dZe dze d<dZedze d <d!Z edze d"<dZ!e dze d#<d$Z"e dze d%<d&Z#e dze d'<dZ$e dze d(<d&Z%e dze d)<dZ&e dze d*<d+Z'ee d,<dZ(e dze d-<d.Z)e dze d/<d0Z*e e+e zdze d1<dZ,ee d2<dZ-ee d3<dZ.edze d4<dZ/edze d5<fd6Z0xZ1S)7 EvollaConfiga protein_encoder_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`SaProtConfig`]. aligner_ffn_mult (`int`, *optional*, defaults to 4): The FFN multiplier for the aligner layer. aligner_enable_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in the aligner layer. aligner_attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities in the aligner layer. aligner_num_add_layers (`int`, *optional*, defaults to 8): The number of additional layers for the aligner layer. resampler_depth (`int`, *optional*, defaults to 6): The depth of the resampler layer in the llama model. resampler_dim_head (`int`, *optional*, defaults to 64): The dimension of the heads in the resampler layer in the llama model. resampler_heads (`int`, *optional*, defaults to 8): The number of heads in the resampler layer in the llama model. resampler_num_latents (`int`, *optional*, defaults to 64): The number of latents in the resampler layer in the llama model. resampler_ff_mult (`int`, *optional*, defaults to 4): The FFN multiplier for the resampler layer. Example: ```python >>> from transformers import EvollaModel, EvollaConfig >>> # Initializing a Evolla evolla-10b style configuration >>> configuration = EvollaConfig() >>> # Initializing a model from the evolla-10b style configuration >>> model = EvollaModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```evollaprotein_encoder_configgANir iri8r rrnum_key_value_headssilu hidden_acti rr rms_norm_epsrope_parametersFattention_biasgattention_dropoutmlp_biasr aligner_ffn_multTaligner_enable_biasr$aligner_attention_probs_dropout_probaligner_num_add_layersresampler_depth@resampler_dim_headresampler_headsresampler_num_latentsresampler_ff_multrrri bos_token_idi  eos_token_id use_cachetie_word_embeddingsr$r%c |j%t|_tjdn4t |jt rtdi|j|_t |di|y)NzX`protein_encoder_config` is `None`. Initializing the `SaProtConfig` with default values.r/)r5r loggerinfo isinstancedictsuper __post_init__)selfkwargs __class__s r1rUzEvollaConfig.__post_init__s\  & & .*6.D ' KKr s 33T :*6*U9T9T*UD ' ''r0)2r&r'r(r) model_typer sub_configs default_thetar5rSrr+r r*rrrrr8r:r-rr;r,r<rr=r.r>r?r@rArBrCrErGrHrIrJrrrKrLlistrMrNr$r%rU __classcell__)rXs@r1r3r3>s# JJ+\:KM=AD#33d:AJK"s"s!!&'t'J#'S'L%48O^d*T18 ND ,/us{T)/Hd#$cDj$'++9<(%$,<)*C$J*"#OS4Z#%'d '"#OS4Z#(*3:*$%sTz%#u##L#*#%L#*%+1L#S /D(1It %%#Jt #',,((r0r3N)r)huggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrr get_loggerr&rPr r3__all__r/r0r1rds!.31,   H %89&#&:&D89P(#P(:P(f  r0