^j!rddlmZddlmZddlmZddlmZedeGdd eZd gZ y ) )strict)PreTrainedConfig)RopeParameters)auto_docstringzUsefulSensors/moonshine-tiny) checkpointceZdZUdZdZdgZdddddZd Zee d <d Z ee d <d Z ee d<dZ ee d<dZ ee d<dZee d<dZee d<dZedze d<dZedze d<dZedze d<dZee d<dZee d<dZee d<dZee d<dZee d<dZee d <dZeezdze d!<dZee d"<d#Z ee d$<d%Z!eeze d&<dZ"edze d'<d(Z#ee$ezdze d)<dZ%edze d*<dZ&ee d+<fd,Z'xZ(S)-MoonshineConfiga encoder_num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `encoder_num_key_value_heads=encoder_num_attention_heads`, the model will use Multi Head Attention (MHA), if `encoder_num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `num_attention_heads`. decoder_num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `decoder_num_key_value_heads=decoder_num_attention_heads`, the model will use Multi Head Attention (MHA), if `decoder_num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `decoder_num_attention_heads`. pad_head_dim_to_multiple_of (`int`, *optional*): Pad head dimension in encoder and decoder to the next multiple of this value. Necessary for using certain optimized attention implementations. encoder_hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder. decoder_hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the decoder. Example: ```python >>> from transformers import MoonshineModel, MoonshineConfig >>> # Initializing a Moonshine style configuration >>> configuration = MoonshineConfig().from_pretrained("UsefulSensors/moonshine-tiny") >>> # Initializing a model from the configuration >>> model = MoonshineModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` moonshinepast_key_valuesdecoder_num_key_value_headsdecoder_num_attention_headsdecoder_num_hidden_layersdecoder_hidden_act)num_key_value_headsnum_attention_headsnum_hidden_layers hidden_acti vocab_sizei  hidden_sizeiintermediate_sizeencoder_num_hidden_layersencoder_num_attention_headsNencoder_num_key_value_headspad_head_dim_to_multiple_ofgeluencoder_hidden_actsiluimax_position_embeddingsg{Gz?initializer_rangedecoder_start_token_idT use_cacherope_parametersis_encoder_decoderFattention_biasgattention_dropout bos_token_id eos_token_id pad_token_idtie_word_embeddingsc |j|j|_|j|j|_|j ddt |di|y)Npartial_rotary_factorg?)rrr r setdefaultsuper __post_init__)selfkwargs __class__s /var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/moonshine/configuration_moonshine.pyr4zMoonshineConfig.__post_init__isX  + + 3/3/O/OD ,  + + 3/3/O/OD ,137 ''))__name__ __module__ __qualname____doc__ model_typekeys_to_ignore_at_inference attribute_maprint__annotations__rrrrrrrr rrstrrr!r"floatr$r%boolr&rdictr'r(r)r*r,listr-r.r4 __classcell__)r7s@r8r r sg& PJ#4"5<<8* MJK!s!%&s&%&s&'(('((.2t2.2t2.2t2$$$$#&S&#u#"#C#It48O^d*T18## ND %(us{( L#* +,L#S /D(,#L#*# $$((r9r N) huggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrr __all__r1r9r8rNsK*/31#9:S(&S(;S(l  r9