^j!,ddlmZddlmZddlmZmZmZmZm Z ddl m Z m Z ddl mZmZddlmZerddlmZmZGd d ed ZGd ded ZeedGddeZdgZy)) BatchFeature) ImageInput)MultiModalDataProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInput)auto_docstringis_vision_available)requires)Emu3ImageProcessorKwargs smart_resizeceZdZUdZeed<y)Emu3TextKwargsa return_for_image_generation (`bool`, *optional*, defaults to `False`): Whether the processed text is intended for image generation tasks. When `True`, the processor prepares inputs for image generation by appending image start tokens and size information to the prompt, and images should not be provided. When `False`, the processor prepares inputs for text generation from images and text, requiring both inputs to be provided. return_for_image_generationN)__name__ __module__ __qualname____doc__bool__annotations__s/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/emu3/processing_emu3.pyrrs"&%rrF)totalc8eZdZUeed<eed<dddddddZy ) Emu3ProcessorKwargs text_kwargs images_kwargsF)rreturn_mm_token_type_idsz1:1i)ratio image_area)r!r"N)rrrrrr _defaultsrrrr r )s/++,1(-   Irr )vision)backendsc eZdZ d fd Ze d dedzdeezeezeezdzde e de fdZ d dZ d Zdefd Z dd ZxZS) Emu3ProcessorNc &|j|_|j|_|j|_|j|_|j |_|j|_|j|_ d|_ t|1|||y)N) chat_template) image_tokenimage_token_id boi_tokenimage_start_token eoi_tokenimage_end_tokenimage_wrapper_tokenfake_token_around_image eof_token bos_tokendownsample_ratiosuper__init__)selfimage_processor tokenizerr-kwargs __class__s rr:zEmu3Processor.__init__;s%00'66!*!4!4(22'0'D'D$",,",, ! )=Qrimagestextr>returnc t|tr|g}n.t|tst|dts td|jt fd|j ji|}|djdd}|djdd }|djd d }|r | td |s| | td i}|j} |j|j} |s| |j|fi|d}t|j} g} |D]} |j | vrt#| }|\}}||j$z}||j$z}||d zz}| |d||j&d|z| }| j)|j |d } |j*| } |j | vr| j-| | D cgc]} | j)d|j  }} nj|rh|j/|||j$\}}| |d||j&}|D cgc]} |j*| |}} ||ggt1|z|d<|djdd }|djdd}|j |fi|ddd i}|j3||dg|r|j5|d|d<t7i|||Scc} wcc} w)aA Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. zAInvalid input text. Please provide a string, or a list of stringstokenizer_init_kwargsr!rFr"r$Nr%zGYou should not provide `images` when `return_for_image_generation=True`zOYou must provide either text or images when `return_for_image_generation=False`r*z image_sizesreturn_tensorsr#image) modalities input_idsmm_token_type_ids)data tensor_type) isinstancestrlist TypeError _merge_kwargsr r= init_kwargspop ValueErrorr1r6r3r<iterrGr.nextr8r5replacer7appendcalculate_generate_sizelen_check_special_mm_tokenscreate_mm_token_type_idsr)r;r@rAr> output_kwargsrr$r%image_featuresimage_start_tokensimage_end_tokensrGprompt_stringssample image_sizeheightwidthimage_seq_lengthimage_placeholder image_promptrHr# text_inputss r__call__zEmu3Processor.__call__Ls& dC 6DD$' 47C0H_` `***  "&.."<"<  '4M&B&F&FGdfk&l#o.227DA"?377 dK &6+=fg g*t|no o $ 6 67"nn-d.B.B-CD+v/A1T11&[M/vhawtOkOkNlm|@PnPmQRbQc)d%#^^D,<,<>OQRSF $/x8F&&&0%%f- .Ucc&FNN?D4D4DEcDc) 88 DLaLabMFE01&5'$B^B^A_`LLPQ&t~~&vh|n=QDQ.4e_,=D ,IN= )'}599:JDQ#0#?#C#CD^`e#f $dnnT_]=-I_Z^_  %%dKWI%N #/3/L/L[YdMe/fK+ ,!BK!B>!BP^__#d Rs ##K6K;c i}|g}|D]\}}t|||jj|jj|jj\}}||j z}||j z}||dzz}|j |dgt|z}|j||dtdi|S)a Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. r)num_image_tokensnum_image_patchesr) rr<spatial_factor min_pixels max_pixelsr8rZr\updater) r;rGr> vision_datarnrfrgrhros r_get_num_multimodal_tokensz(Emu3Processor._get_num_multimodal_tokenss  "! !, :  ,((77((33((33 !  4#8#88!6!66#)UQY#7  ''(89 :"#c+&6 6    4D[lm n, ,,rctt|jd\}}||z}||z dz}tt||z|z }tt||z|z } || fS)N:g?)mapintsplitround) r;r$r%rprgrf current_area target_ratio token_height token_widths rr[z%Emu3Processor.calculate_generate_sizespCS!12 vv~ "\1c9 5,!6!GHI % 4~ EFG [((rc <|jj|fi|SN)r< postprocess)r;r@r>s rrzEmu3Processor.postprocesss /t##//A&AArc ||dk(r|j|fd|i|S|dk(r|j|d}|dSt|jjd|d) a  Post-process the output of a multimodal model to return the requested modality output. If the model cannot generated the requested modality, an error will be raised. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. generation_mode (`str`, *optional*): Generation mode indicated which modality to output and can be one of `["text", "image", "audio"]`. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `list[Union[str, PIL.Image.Image]]`: The decoded text or generated image. rAskip_special_tokensrIzPIL.Image.Image)rH pixel_valuesz# got an unexpected generation_mode=z.. Supported options are only `text` and `image)post_process_image_text_to_textrrVr?r)r;generated_outputsrgeneration_moder>r@s rpost_process_multimodal_outputz,Emu3Processor.post_process_multimodal_outputs*  "o&?7477!7JNT  '%%&7HY%ZF.) )>>**++NN_`NO rr)NN)TN)rrrr:r rr r rQr r rrlrur[rr __classcell__)r?s@rr*r*8s R"%)aeM`T!M`++d9o=EV@WWZ^^M`,- M`  M`M`^ -D)B*BLP!rr*N)image_processing_utilsr image_utilsrprocessing_utilsrrrrr tokenization_utils_baser r utilsr r utils.import_utilsrimage_processing_emu3rrrr r*__all__rrrrsz"3%ddC8*M &Zu & *%  ;qNq qh  r