^jdZddlmZddlmZddlmZmZmZm Z ddl m Z m Z ddl mZmZej eZdZGdd ed ZGd d ed ZeGddeZdgZy)z Processor class for Janus. ) BatchFeature) ImageInput)ProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInput)auto_docstringloggingzYou are a helpful language and vision assistant. You are able to understand the visual content that the user provides, and assist the user with a variety of tasks using natural language. ceZdZUdZeed<y)JanusTextKwargsas generation_mode (`str`, *optional*, defaults to `"text"`): The generation mode indicating which modality to generate. Can be one of `"text"` or `"image"`. When set to `"text"`, the processor prepares inputs for text generation. When set to `"image"`, it prepares inputs for image generation by appending image start tokens to the prompt. generation_modeN)__name__ __module__ __qualname____doc__str__annotations__u/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/transformers/models/janus/processing_janus.pyrr"srrF)totalc.eZdZUeed<ddddddidZy ) JanusProcessorKwargs text_kwargsFlefttext)padding padding_siderreturn_tensorspt)r common_kwargsN)rrrrr _defaultsrrrrr-s"  #(&U[\*D1Irrc eZdZeZ ddedzdedeffd Ze dde e ze e ze e zde dzdeed effd Zd ed ed efd Zde fdZ ddZxZS)JanusProcessorN chat_templateuse_default_system_promptnum_image_tokensc ||_|j|_|j|_|j|_||_t|!|||y)a use_default_system_prompt (`bool`, *optional*, defaults to `False`): Use default system prompt for Text Generation. num_image_tokens (`int`, *optional*, defaults to `576`): The number of placeholder image tokens needed per one image. )r'N) r) image_token boi_tokenimage_start_token eoi_tokenimage_end_tokenr(super__init__)selfimage_processor tokenizerr'r(r)kwargs __class__s rr1zJanusProcessor.__init__9sS!1$00!*!4!4(22)B& )=Qrrimagesr5returnc X|jtfd|jji|}|dj d}|j r|dk(r|Dcgc] }t |}}n"|dk(r|Dcgc]}||j}}t|$d||d|}|Scc}wcc}w)aA Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. tokenizer_init_kwargsrrrimage)r7rr) _merge_kwargsrr4 init_kwargspopr(DEFAULT_SYSTEM_PROMPTr-r0__call__) r2rr7r5 output_kwargsrsample model_inputsr6s rr@zJanusProcessor.__call__Ps$+**  8<8R8R V\ ( 6::;LM  ) )o.GEIJ6,-fX6JDJ  'FJKFvht5567KDKw'RvDRMR  KKs B"5B' image_inputs image_idxcl|j|j|jzz|jz}|S)N)r-r+r)r/)r2rDrEone_img_tokenss rreplace_image_tokenz"JanusProcessor.replace_image_tokenos5//43C3CdF[F[3[\_c_s_ssrc <|jj|fi|S)z Forwards all arguments to the image processor's `postprocess` method. Refer to the original method's docstring for more details. )r3 postprocess)r2r7r5s rrJzJanusProcessor.postprocessss" 0t##//A&AArc ||dk(r|j|fd|i|S|dk(r1t|j}|j|d}|dSt |j j d|d) a  Post-process the output of a multimodal model to return the requested modality output. If the model cannot generated the requested modality, an error will be raised. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. generation_mode (`str`, *optional*): Generation mode indicated which modality to output and can be one of `["text", "image", "audio"]`. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `list[Union[str, PIL.Image.Image]]`: The decoded text or generated image. rskip_special_tokensr;zPIL.Image.Image)r! pixel_valuesz# got an unexpected generation_mode=z.. Supported options are only `text` and `image)post_process_image_text_to_textlistfloatrJ ValueErrorr6r)r2generated_outputsrLrr5r7s rpost_process_multimodal_outputz-JanusProcessor.post_process_multimodal_outputzs*  "o&?7477!7JNT  ' $%6%<%<%> ? %%&7HY%ZF.) )>>**++NN_`NO r)NFi@)NN)TN)rrrrvalid_processor_kwargsrboolintr1r r r rOrrrr@dictrHrJrS __classcell__)r6s@rr&r&5s1 %)*/ # RTz R $( R  R.[_$(++d9o=EV@WWT!-.   <B*BLP"rr&N)rfeature_extraction_utilsr image_utilsrprocessing_utilsrrrrtokenization_utils_baser r utilsr r get_loggerrloggerr?rrr&__all__rrrras5%TTC,   H %Nj+5f^ffR  r