^jk'dZddlmZddlmZddlmZddlmZm Z ddl m Z m Z e jeZeGdd eZeGd d eZeGd d eZedeGddeZdZdgZy)zESM model configuration)Union)strict)PreTrainedConfig)auto_docstringlogging)intervalis_divisible_byc>eZdZUdZdZedzed<dZedzed<dZedzed<dZ edzed <d Z edzed <d Z edzed <dZ edzed<dZ edzed<dZedzed<dZedzed<dZedzed<dZedzed<dZedzed<dZedzed<dZedzed<y)StructureModuleConfiga Args: sequence_dim: Single representation channel dimension pairwise_dim: Pair representation channel dimension ipa_dim: IPA hidden channel dimension resnet_dim: Angle resnet (Alg. 23 lines 11-14) hidden channel dimension num_heads_ipa: Number of IPA heads num_qk_points: Number of query/key points to generate during IPA num_v_points: Number of value points to generate during IPA dropout_rate: Dropout rate used throughout the layer num_blocks: Number of structure module blocks num_transition_layers: Number of layers in the single representation transition (Alg. 23 lines 8-9) num_resnet_blocks: Number of blocks in the angle resnet num_angles: Number of angles to generate in the angle resnet trans_scale_factor: Scale of single representation transition hidden dimension epsilon: Small number used in angle resnet normalization inf: Large number used for attention masking iN sequence_dim pairwise_dimipa_dim resnet_dim num_heads_ipa num_qk_points num_v_points皙? dropout_rate num_blocksnum_transition_layersnum_resnet_blocks num_angles trans_scale_factorg:0yE>epsilongj@inf)__name__ __module__ __qualname____doc__r int__annotations__rrrrrrrfloatrrrr!r#r$r%t/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/esm/configuration_esm.pyr r s D #L#*""L#*"GS4Z Jd  "M3:" !M3:! L#* !$L%$,$Jd ()3:)$%sTz%Jd %'d ' GUT\ Cr.r ceZdZUdeiZdZedzed<dZedzed<e dd Z edzed <d Z edzed <d Z edzed<d Z edzed<edd Zeezdzed<dZeezdzed<dZedzed<edd Zedzed<d Zedzed<dZeedfdzed<fdZdZxZS) TrunkConfigstructure_module0Nrisequence_state_dimr)divisorr)defaultpairwise_state_dim sequence_head_widthpairwise_head_width position_binsg?)maxdropout layer_dropFcpu_grad_checkpointr)minr max_recycles chunk_sizer c |jt|_n4t|jtrtdi|j|_t |di|yNr-)r2r isinstancedictsuper __post_init__selfkwargs __class__s r/rIzTrunkConfig.__post_init__bsP  ($9$;D ! --t 4$9$RDr,r?r@boolrBrCr2rrGrIrS __classcell__rMs@r/r1r1Qs%'<=KJd %)d )%?_Q%?%Ld L&(t(&(t( "M3:""3(s"3C"@GUS[4 @%(J d"(',,.xAq9L#*9 Jd  DHeD"99:TAH(r.r1ceZdZUdeiZdZedzed<dZe dzed<dZ e dzed<dZ e dzed<dZ e dzed <d Z eezdzed <dZe dzed <dZe dzed <dZedzed<dZeedfdzed<fdZxZS) EsmFoldConfigtrunkNesm_typeTfp16_esmFuse_esm_attn_mapesm_ablate_pairwiseesm_ablate_sequencer=esm_input_dropoutembed_aa bypass_lmrlddt_head_hid_dimr1c |jt|_n4t|jtrtdi|j|_t |di|yrE)rZr1rFrGrHrIrJs r/rIzEsmFoldConfig.__post_init__sG :: $DJ  D )$2tzz2DJ ''r.)r&r'r(r1rTr[strr+r\rUr]r^r_r`r,r*rarbrcrZrrGrIrVrWs@r/rYrYsK(KHcDj HdTk $)dTk)',,',,,/us{T)/ HdTk "Itd{"$'sTz'/3E5}$ % ,3((r.rYzfacebook/esm-1b) checkpointceZdZUdZdZdeiZdZedze d<dZ edze d<dZ edze d<dZ ee d <d Z ee d <d Zee d <d Zee d<dZedze d<dZedze d<dZee d<dZee d<dZee d<dZedze d<dZedze d<dZee d<dZedze d<dZedze d <dZedze d!<dZeezdze d<dZ e!ee"ed"fzdze d#<dZ#edze d$<dZ$edze d%<dZ%ee d&<dZ&edze d'<d(Z'ee!ezdze d)<fd*Z(xZ)S)+ EsmConfiga mask_token_id (`int`, *optional*): The index of the mask token in the vocabulary. This must be included in the config because of the "mask-dropout" scaling trick, which will scale the inputs depending on the number of masked tokens. rope_theta (`float`, defaults to 10000.0): The base period of the RoPE embeddings. Only used when `position_embedding_type` is set to `"rotary"`. position_embedding_type (`str`, *optional*, defaults to `"absolute"`): Type of position embedding. Choose either `"absolute"` or "rotary"`. emb_layer_norm_before (`bool`, *optional*): Whether to apply layer normalization after embeddings but before the main stem of the network. token_dropout (`bool`, defaults to `False`): When this is enabled, masked tokens are treated as if they had been dropped out by input dropout. is_folding_model (`bool`, defaults to `False`): When this is enabled, ESMFold model will be initialized. esmfold_config (`dict`, *optional*): Configuration to initiate the ESMFold module. vocab_list (`list`, *optional*): List of the vocabulary items. Examples: ```python >>> from transformers import EsmModel, EsmConfig >>> # Initializing a ESM facebook/esm-1b style configuration >>> configuration = EsmConfig(vocab_size=33) >>> # Initializing a model from the configuration >>> model = EsmModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```esmesmfold_configN vocab_size mask_token_id pad_token_idi hidden_sizernum_hidden_layersnum_attention_headsi intermediate_sizerhidden_dropout_probattention_probs_dropout_probimax_position_embeddingsg@ rope_thetag{Gz?initializer_rangeg-q=layer_norm_epsabsoluteposition_embedding_typeT use_cacheemb_layer_norm_beforeF token_dropoutis_folding_model. vocab_list is_decoderadd_cross_attentiontie_word_embeddings bos_token_idr eos_token_idc |jr|j%tjdt |_n4t |jt rt di|j|_|j3tjdt|_nd|_d|_|j"t|jddr tdt|4di|y)NzCNo esmfold_config supplied for folding model, using default values.zHNo vocab_list supplied for folding model, assuming the ESM-2 vocabulary!r]FzOThe HuggingFace port of ESMFold does not support use_esm_attn_map at this time!r-)r}rjloggerinforYrFrGr~warningget_default_vocab_listgetattrrPrHrIrJs r/rIzEsmConfig.__post_init__s  ""* ab&3o#D//6&3&Jd6I6I&J#&ij"8":"&D "DO    *wt7J7JL^`e/fno o ''r.)*r&r'r(r) model_typerYrTrkr*r+rlrmrnrorprqrrr,rsrtrurvrwryrerzrUr{r|r}rjrGr~listtuplerrrrrrIrVrWs@r/rhrhs  DJ#]3K!Jd ! $M3:$#L#*#Ks!!!s!(++14 %$,4#'S'J#u##(NEDL(*4S4Z4It)-4$;-!&M4$;&$)dTk)26ND=(4/659JS E#s(O+d29#Jt #',, $$#L#*#+,L#S /D(,((r.rhcy)N)!zzzzLAGVSERTIDPKQNFYMHWCXBUZOrO-zzr-r-r.r/rrs" r.N)r)typingrhuggingface_hub.dataclassesrconfiguration_utilsrutilsrrutils.type_validatorsr r get_loggerr&rr r1rYrhr__all__r-r.r/rs.3,>   H %1,11h/"//d($((,,-R( R(.R(j#L -r.