^j1!dZddlmZddlmZddlmZddlmZddl m Z ed eGd d eZ ed eGd deZ dd gZ y)zMoshi model configuration)strict)PreTrainedConfig)RopeParameters)auto_docstring) AutoConfigzkmhf/hf-moshiko) checkpointceZdZUdZdZdgZdZeed<dZ eed<dZ eed <d Z eed <d Z eed <dZ edzed<dZeed<dZeed<dZeed<dZedzed<dZeed<dZeed<dZeed<dZeezed<dZeed <d!Zeed"<dZeed#<d$Zeed%<dZedzed&<dZedzed'<dZee ezdzed(<fd)Z!d*Z"xZ#S)+MoshiDepthConfiga input_size (`int`, *optional*, defaults to 4096): Dimensionality of the input hidden states. Used to connect the main decoder to the depth decoder. audio_vocab_size (`int`, *optional*, defaults to 2048): Vocabulary size of the audio part of model. Defines the number of different tokens that can be represented by the `audio_codes` passed when calling the Moshi models. ffn_dim (`int`, *optional*, defaults to 5632): Dimensionality of the "intermediate" (often named feed-forward) layer in the depth decoder block. Must be even. Example: ```python >>> from transformers import ( ... MoshiDepthConfig, ... MoshiDepthDecoder, ... ) >>> configuration = MoshiDepthConfig() >>> # Initializing a MoshiDepthDecoder (with random weights) from the kmhf/hf-moshiko style configuration >>> model = MoshiDepthDecoder(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` moshi_depthpast_key_values} vocab_sizei hidden_size input_sizenum_hidden_layersnum_attention_headsNnum_key_value_headsiaudio_vocab_size max_position_embeddingssilu hidden_acthead_dim{Gz?initializer_rangeT use_cachesliding_windowattention_dropoutiffn_dim:0yE> rms_norm_eps num_codebooksFtie_word_embeddings pad_token_id bos_token_id eos_token_idc |j |jn |j|_|jxs|j|jz|_t |di|y)N)rrrrsuper __post_init__)selfkwargs __class__s x/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/moshi/configuration_moshi.pyr1zMoshiDepthConfig.__post_init__Ns[(,(@(@(LD $ $RVRjRj   U)9)9T=U=U)U  ''cZ|jdzdk(rtd|jdy)OPart of `@strict`-powered validation. Validates the architecture of the config.r `ffn_dim=` must be even.N)r& ValueErrorr2s r5validate_architecturez&MoshiDepthConfig.validate_architectureUs0 < __classcell__r4s@r5r r s 4J#4"5JKJs!!&*t* c #$S$JHcDj#u#ItNC%(us{(GSL%M3 %%#L#*##L#*#+/L#S /D(/(Hr6r ceZdZUdZdZdgZeedZdZ e e d<dZ e e d<d Z e e d <d Ze e d <d Ze d ze d <d Ze d ze d<dZe e d<d Zeezd ze d<dZee d<d Ze d ze d<dZee d<dZee d<dZe e d<dZee ze d<dZe e d<dZee d<d Z e e d!<d"Z!ee d#<d Z"e d ze d$<d Z#e d ze d%<d Z$e e%e zd ze d&<d Z&ee'zd ze d'<d Z(ee'zd ze d(<fd)Z)d*Z*e+d+Z,e-d'e'fd,Z.xZ/S)- MoshiConfiga audio_vocab_size (`int`, *optional*): Vocabulary size of the audio part of model. Defines the number of different tokens that can be represented by the `audio_codes` passed when calling the Moshi models. ffn_dim (`int`, *optional*, defaults to 22528): Dimensionality of the "intermediate" (often named feed-forward) layer in the main decoder block. Must be even. audio_encoder_config (`PreTrainedConfig | dict`, *optional*): Configuration for the audio encoder. depth_decoder_config (`PreTrainedConfig | dict`, *optional*): Configuration for the depth decoder. Example: ```python >>> from transformers import ( ... MoshiConfig, ... MoshiForConditionalGeneration, ... ) >>> configuration = MoshiConfig() >>> # Initializing a MoshiForConditionalGeneration (with random weights) from the kmhf/hf-moshiko style configuration >>> model = MoshiForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> # Saving the model, including its configuration >>> model.save_pretrained("kmhf/hf-moshiko") >>> # loading model and config from pretrained folder >>> moshi_config = MoshiConfig.from_pretrained("kmhf/hf-moshiko") >>> model = MoshiForConditionalGeneration.from_pretrained("kmhf/hf-moshiko", config=moshi_config) ```moshir)audio_encoder_configdepth_decoder_configrrrr rrNrri rrope_parametersrrrrr Tr!r#r$r%iXr&r'r(r"r)Fr*r+r,r-rPrQc h|j |jn |j|_|jxs|j|jz|_t |j t rB|j jdd}tj|fi|j |_n&|j tjd|_|j|j jn |j|_ t |jt rc|jj|j|j|j|jdt!di|j|_ n|jt!|_ t#|Hdi|y)NrCmimi)rrrr)r/)rrrr isinstancerPdictpopr for_modelr codebook_sizerQupdaterr)r r0r1)r2r3audio_encoder_model_typer4s r5r1zMoshiConfig.__post_init__sm(,(@(@(LD $ $RVRjRj   U)9)9T=U=U)U d// 6'+'@'@'D'D\SY'Z $(2(<(<=U(sY]YrYr(sD %  & & .(2(<(z!MoshiConfig.validate_architectures <propertyr_ classmethodrcrKrLs@r5rNrN[s! FJ#4"5+5O_`KJKs!!&*t*#'cDj'#'S'48O^d*T18JHcDj#u#ItNC%(us{(GSL%M3 %%#L#*##L#*#+/L#S /D(/;?$!11D8?;?$!11D8?(<77 .  r6rNN)rBhuggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrauto.configuration_autor r rN__all__r/r6r5rms .31#0,->H'>H.>HB,-| "| .| ~ , -r6