^j5UddlmZddlmZmZddlmZddlmZddl m Z m Z ddl m Z ddlmZmZmZej$eZiiiiiiiiiiiiiidZeeeeeffed <ed eeefd eeeffd Zed eed eefd Zd eeefeezd eeefeezfdZGddZGddeZGddeZGddeZedded edfdZdeded efdZ y)) lru_cache)Anyoverload) constants)InferenceProviderMapping) MimeBytesRequestParameters)ChatCompletionInputMessage)build_hf_headers get_tokenlogging)cerebrascohere deepinfrazfal-aiz fireworks-aigroqz hf-inferencenscaleovhcloud replicatescalewaytogether wavespeedzzai-org!HARDCODED_MODEL_INFERENCE_MAPPINGobjreturncyNrs w/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/huggingface_hub/inference/_providers/_common.py filter_noner *s8;cyrrrs rr r ,s.1r!cft|trEi}|jD].\}}| t|ttfr t |}|||<0|St|tr1|Dcgc]%}t|ttfr t |n|'c}St dt |cc}w)NzExpected dict or list, got ) isinstancedictitemslistr ValueErrortype)rcleanedkvs rr r 0s#t"$IIK DAqy!dD\*NGAJ   #tNQR*Qt "= A1DRR 249+> ??Ss**B.ceZdZdZdedededdfdZddd ed eeefd ed edzd edzdeeefdzdefdZ d de ezdedzdefdZ d edzdefdZ d edzde fdZd eeefdeeefdzdedzdeeeffdZd ed edeeeffdZd ededefdZd edefdZded edefdZd ed ede dedzfdZd ed ede dedzdedzf dZy)!TaskProviderHelperz.Base class for task-specific provider helpers.providerbase_urltaskrNc.||_||_||_yr)r/r1r0)selfr/r0r1s r__init__zTaskProviderHelper.__init__Ds     r!) extra_payloadinputs parametersheadersmodelapi_keyr5c|j|}|j|}|j||}|j||j}|j |||} | t | t|xsi} |j||||} | | td| | td|j|| | } t||j|j| | | S)z Prepare the request to be sent to the provider. Each step (api_key, model, headers, url, payload) can be customized in subclasses. )provider_mapping_infoz8Both payload and data cannot be set in the same request.z2Either payload or data must be set in the request.)urlr1r9jsondatar8) _prepare_api_key_prepare_mapping_info_prepare_headers _prepare_url provider_id_prepare_payload_as_dictrecursive_merger _prepare_payload_as_bytesr(_normalize_headersr r1) r3r6r7r8r9r:r5r<r=payloadr?normalized_headerss rprepare_requestz"TaskProviderHelper.prepare_requestIs ''0!% : :5 A''9)>)J)JK// Zo/p  %g{=;NB/OPG--fjBWYfg  4#3WX X ?t|QR R"44WgtL '33&   r!responserequest_paramsc|S)z Return the response in the expected format. Override this method in subclasses for customized response handling.r)r3rLrMs r get_responsezTaskProviderHelper.get_response~s r!cT| t}|td|jd|S)zZReturn the API key to use for the request. Usually not overwritten in subclasses.z)You must provide an api_key to work with z$ API or log in with `hf auth login`.)r r(r/r3r:s rr@z#TaskProviderHelper._prepare_api_keys: ?kG ?;DMM?Jno r!c |td|jdtj|jij|rt|j|Sd}t |D]}|j|jk(s|}n|td|d|jd|j |j k7r6td|d|j d|jd|j d |j d k(r&tjd|d |jd |j d k(r&tjd |d|jd|S)zbReturn the mapped model ID to use for the request. Usually not overwritten in subclasses.Nz+Please provide an HF model ID supported by .zModel z is not supported by provider z is not supported for task z and provider z. Supported task: stagingz! is in staging mode for provider z. Meant for test purposes only.errorz,Our latest automated health check on model 'z' for provider 'z<' did not complete successfully. Inference call might fail.) r(r/rget!_fetch_inference_provider_mappingr1statusloggerwarning)r3r9provider_mappingmappings rrAz(TaskProviderHelper._prepare_mapping_infos =J4==/YZ[\ \ - 0 0 C G G N4T]]CEJ J8? G4==0#*    #veW,J4==/YZ[\ \  DII -:499+^TXTaTaSbc##3#8#8"9<   " "i / NN@Ono   " "g - NN>ugEUVZVcVcUde--  r!rIr?c|jDcic]\}}| |j|}}}|jd&||j|j|d<|S|d|d<|Scc}}w)zzNormalize the headers to use for the request. Override this method in subclasses for customized headers. z content-typezapplication/json)r&lowerrV mime_type)r3r8rIr?keyvaluerJs rrHz%TaskProviderHelper._normalize_headerssDK==?hZS%V[Vgciik50hh  ! !. 1 9DNN$>59^^">2"!$5G">2!! is A.A.c"it||S)zwReturn the headers to use for the request. Override this method in subclasses for customized headers. )token)r )r3r8r:s rrBz#TaskProviderHelper._prepare_headerss >"1=W==r! mapped_modelc|j|}|j||}|jdd|jdS)zVReturn the URL to use for the request. Usually not overwritten in subclasses./)_prepare_base_url_prepare_routerstriplstrip)r3r:rdr0routes rrCzTaskProviderHelper._prepare_urlsK))'2##L'://#&'qc):(;<>s>tCH~>=C=s=s= ! ! !3'+D\     8  d{  T  r!r.c jeZdZdZdedeffd ZdededefdZd eee zd ed e ded zfd Z xZ S)BaseConversationalTaskz Base class for conversational (chat completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/chat r/r0c*t|||dy)Nconversationalr/r0r1superr4r3r/r0 __class__s rr4zBaseConversationalTask.__init__ s (XDTUr!rdr:rcy)Nz/v1/chat/completionsrrus rrhz%BaseConversationalTask._prepare_route s%r!r6r7r<Nc<td|i|d|jiS)Nmessagesr9r rDrws rrEz/BaseConversationalTask._prepare_payload_as_dicts' Ji*igG\GhGhijjr!) ryrzr{r|r}r4rhr'r%r rrE __classcell__rs@rrrsw VVV&3&&&kT667kk 8 k  kr!rcHeZdZdZfdZdedefdZdedzdefdZxZ S) AutoRouterConversationalTaska Auto-router for conversational tasks. We let the Hugging Face router select the best provider for the model, based on availability and user preferences. This is a special case since the selection is done server-side (avoid 1 API call to fetch provider mapping). c(t|ddy)Nautozhttps://router.huggingface.co)r/r0r)r3rs rr4z%AutoRouterConversationalTask.__init__!s &3RSr!r:rcR|jds td|jS)rmrnz>Cannot select auto-router when using non-Hugging Face API key.)ror(r0rQs rrgz.AutoRouterConversationalTask._prepare_base_url$s( !!%(]^ ^== r!r9Nc<| tdtd||ddS)z In auto-router, we don't need to fetch provider mapping info. We just return a dummy mapping info with provider_id set to the HF model ID. zPlease provide an HF model ID.rliver)r/ hf_model_id providerIdrXr1)r(r)r3r9s rrAz2AutoRouterConversationalTask._prepare_mapping_info.s1 ==> >'!   r!) ryrzr{r|r4r}rgrrArrs@rrrs9T!!! 3: :R r!rc ^eZdZdZdedeffd ZdededefdZd ed ed e ded zfd Z xZ S)BaseTextGenerationTaskz Base class for text-generation (completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/completions r/r0c*t|||dy)Nztext-generationrrrs rr4zBaseTextGenerationTask.__init__Es (XDUVr!rdr:rcy)Nz/v1/completionsrrus rrhz%BaseTextGenerationTask._prepare_routeHs r!r6r7r<Nc<td|i|d|jiS)Npromptr9rrws rrEz/BaseTextGenerationTask._prepare_payload_as_dictKs'Hfg gGEZEfEfghhr!) ryrzr{r|r}r4rhrr%rrErrs@rrr?sf WWW!3!!!ii'+iD\i ir!rN)maxsizer9rc|ddlm}|j|dg}|j}|t d||S)z; Fetch provider mappings for a model from the Hub. r)HfApiinferenceProviderMapping)expandz$No provider mapping found for model )huggingface_hub.hf_apir model_infoinference_provider_mappingr()r9rrpr[s rrWrWQsM - 7  e-G,H  ID66?wGHH r!dict1dict2c i||jDcic]>\}}|||vr2t||trt|trt|||n|@c}}Scc}}wr)r&r$r%rF)rrr`ras rrFrF_sx    $kkm  U u E#J!=*UTXBY!sU3    sAA)! functoolsrtypingrrhuggingface_hubrrr!huggingface_hub.inference._commonrr :huggingface_hub.inference._generated.types.chat_completionr huggingface_hub.utilsr r r get_loggerryrYrr%r}__annotations__r r'r.rrrrWrFrr!rrs %;JaFF   H % +U!4T#7O2O-P(P#Q2 ;T#s(^;S#X; ; 1T#Y1491 1@T#s(^d3i/@DcNT#Y4N@"@@Fk/k*# #9# Li/i$ 4 S T:T5U   4   r!