^j D?" dZddlZddlmZddlZddlmZmZddlm Z m Z ddl m Z m Z mZmZmZmZmZmZmZmZddlmZmZdd lmZed Zed Zeeed fZeedzed fZeedzed fZ ejCdddg dkde deddfdZ"ejCddgdddddddddddddddddddddedeeed fdeeed fd eeed! fd"eeed# fd$eeed% fd&eeed' fd(eeed) fde d*eedzed+ fded,ee#ed- fd.ee#ed/ fd0ee#dzed1 fd2eedzed3 fd4ee$dzed5 fd6ed7eedzed8d9 fd:eedzed; fdeedzed?d@ fdAeedzedBdC fdDe dEe dFedGedHeedzedIeedJKfddf8dLZ%ejCddMg dldeeed fded eedzedN fde deddf dOZ& dmdeddfdPZ'ejCddQge'ejCdRdSTe'ejQedUVejCdWg dkdede deddfdXZ)ejCdYg dndede deedzed fd eedzed! fd"eedzed# fd$eedzed% fdeedzedZ fd6eedzed[ fd*eedzed+ fd,ee#dzed- fd.ee#dzed/ fd0ee#dzed1 fd2eedzed3 fd4ee$dzed5 fdeddf d\Z*ejCd]g dodede d^ee+ed_d` fdeddf daZ,ejCdbg dkdede deddfdcZ-ejCddg dodede deee+edfdg fdeddf dhZ.ejCdig dkdede deddfdjZ/y)pz2CLI commands for Hugging Face Inference Endpoints.N) Annotated)InferenceEndpointScalingMetricInferenceEndpointType)CLIErrorHfHubHTTPError) EnvFileOptEnvOpt RevisionOptSecretsFileOpt SecretsOpt SoftChoiceTokenOpt get_hf_api parse_env_map typer_factory)ArgumentOption)outz(Manage Hugging Face Inference Endpoints.)helpz.Interact with the Inference Endpoints catalog.zEndpoint name.z_The namespace associated with the Inference Endpoint. Defaults to the current user's namespace.z list | lszhf endpoints lsz"hf endpoints ls --namespace my-org)examples namespacetokenreturnct|} |j||}g}|D]o}|j}|jdi}|jdi} |jdi} |jd i} |j|jd d t| tr| jd d nd t|tr|jd d nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd t| tr| jdd nd drtj|d y#t$rR}tjd|t j j|jj|d}~wwxYw)z6Lists all Inference Endpoints for the given namespace.r)rrzListing failed: codeNstatusmodelcomputeprovidername repositorystatetask framework instanceTypevendorregion)r#r rr'r(instancer*r+)id_key)rlist_inference_endpointsrrerrorclick exceptionsExitresponse status_coderawgetappend isinstancedicttable) rrapi endpointsr/resultsendpointr5rr r!r"s r/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/huggingface_hub/cli/inference_endpoints.pylsr@1s 5 !CP009E0R G ll2&$'')R(77:r*+8B5$8O<4UW5?5M&**Wb1SU1;E41H &"-b;EeT;RUYY{B7XZ?I'SW?XGKK;^`8B8T8R(,,x4XZ8B8T8R(,,x4XZ   $IIgf%/ P $UG,-##)C)C#D%OPsF00 H 9A HH deployzChf endpoints deploy my-endpoint --repo gpt2 --framework pytorch ...)r#r)rr'r min_replica max_replicascale_to_zero_timeoutscaling_metricscaling_thresholdrevision custom_image health_routeportcontainer_commandcontainer_argsenvenv_filesecrets secrets_file endpoint_typer#repozeThe name of the model repository associated with the Inference Endpoint (e.g. 'openai/gpt-oss-120b').r(z@The machine learning framework used for the model (e.g. 'vllm'). acceleratorz?The hardware accelerator to be used for inference (e.g. 'cpu'). instance_sizezNThe size or type of the instance to be used for hosting the model (e.g. 'x4'). instance_typezYThe cloud instance type where the Inference Endpoint will be deployed (e.g. 'intel-icl').r+zTThe cloud region in which the Inference Endpoint will be created (e.g. 'us-east-1').r*zVThe cloud provider or vendor where the Inference Endpoint will be hosted (e.g. 'aws').r'zCThe task on which to deploy the model (e.g. 'text-classification').rBzVThe minimum number of replicas (instances) to keep running for the Inference Endpoint.rCzRThe maximum number of replicas (instances) to scale to for the Inference Endpoint.rDzFThe duration in minutes before an inactive endpoint is scaled to zero.rEz!The metric reference for scaling.rFzeThe scaling metric threshold used to trigger a scale up. Ignored when scaling metric is not provided.rGrHz--custom-imagezfDocker image URL for a custom container (e.g. 'nexagi/sglang:v0.5.12'). Requires '--framework custom'.rIz]Health check route exposed by the custom container (e.g. '/health'). Requires --custom-image.rJzKPort the custom container listens on (e.g. 30000). Requires --custom-image.rKz--container-commandzOverride the container entrypoint, as a quoted string split into tokens (e.g. "python -m sglang.launch_server"). Requires --custom-image.rLz--container-argszArguments appended to the container entrypoint, as a quoted string split into tokens (e.g. "--tp 8 --reasoning-parser qwen3"). Requires --custom-image.rMrNrOrPrQz--typezTEndpoint access type. Defaults to 'authenticated' (token-gated, publicly reachable).) click_typerc||||s|r tdd}|d|i}|||d<|||d<t||jDcic] \}}||xsd}}}t||jDcic] \}}||xsd}}}i} ||| d<||| d<|rtj|| d <|rtj|| d <|r|| d <|r|| d <t | }!|!j d!id|d|d|d|d|d|d|d|d|d| d| d| d| d|d|d| d|| }"tj|"jtjd|d ycc}}wcc}}w)"z3Deploy an Inference Endpoint from a Hub repository.NzX--health-route, --port, --container-command and --container-args require --custom-image.url healthRouterJr$typerHrKrLrMrOrr#r%r(rSrTrUr+r*rr'rrBrCrErFrDrGzUse 'hf endpoints describe z!' to check the deployment status.) rritemsshlexsplitrcreate_inference_endpointrr9r5hint)#r#rRr(rSrTrUr+r*rr'rrBrCrDrErFrGrHrIrJrKrLrMrNrOrPrQcustom_image_dictkeyvalueenv_map secrets_mapparamsr;r>s# r?rArATs&V!9T=MQbftqrr%)"L1  #/; m ,  (, f %2?X2N2T2T2VWJCsEKRWGW6CG\6Z6`6`6bc U3 #cKcF &v$!2~&+kk2C&D"##(;;~#> u 'y 5 !C,s,,   $  $   &, 4!" %H(HHX\\HH*4&0Q RSQXcs E*:E0zChf endpoints catalog deploy --repo meta-llama/Llama-3.2-1B-InstructzPThe hardware accelerator to be used for inference (e.g. 'cpu', 'gpu', 'neuron').cDt|} |j|||||}tj|jy#t$rR}tjd|t j j|jj|d}~wwxYw)z4Deploy an Inference Endpoint from the Model Catalog.r)repo_idr#rSrrzDeployment failed: rN) r&create_inference_endpoint_from_catalogrrr/r0r1r2r3r4r9r5)rRr#rSrrr;r>r/s r?deploy_from_catalogrjs& 5 !C P==# > HHX\\ P 'w/0##)C)C#D%OPA B A BBc,t|} |j|}tjd|iy#t$rR}tjd|t j j|jj|d}~wwxYw)zList available Catalog models.rzCatalog fetch failed: rNmodels) rlist_inference_catalogrrr/r0r1r2r3r4r9)rr;rmr/s r? list_catalogro7s 5 !CP++%+8 HHh  P *5'23##)C)C#D%OPs8 BA BBzhf endpoints catalog lsz list-catalogT)r#hiddencatalogr#z!hf endpoints describe my-endpointc@t|} |j|||}tj|jy#t$rR}tjd|t j j|jj|d}~wwxYw)z+Get information about an existing endpoint.rr#rrzFetch failed: rN) rget_inference_endpointrrr/r0r1r2r3r4r9r5r#rrr;r>r/s r?describerwLs 5 !CP--49TY-Z HHX\\ P N5'*+##)C)C#D%OPA B A BBz/hf endpoints update my-endpoint --min-replica 2zBThe machine learning framework used for the model (e.g. 'custom').zrThe specific model revision to deploy on the Inference Endpoint (e.g. '6c0e6080953db56375760c0471a8c5f2929baf11').cXt|} |j|||||||||| | | | | |}tj|jy#t$rR}tjd|t j j|jj|d}~wwxYw)zUpdate an existing endpoint.r)r#rr%r(rGr'rSrTrUrBrCrDrErFrzUpdate failed: rN) rupdate_inference_endpointrrr/r0r1r2r3r4r9r5)r#rrRrSrTrUr(rGr'rBrCrDrErFrr;r>r/s r?updater{]s^ 5 !CP00#''##"7)/1 (HHX\\ P OE7+,##)C)C#D%OPs A B)A B$$B)zhf endpoints delete my-endpointyesz--yeszSkip confirmation prompts.cntjd|d|t|} |j|||tjd |d | y#t$rR}tj d|t jj|jj|d}~wwxYw) z)Delete an Inference Endpoint permanently.zDelete endpoint 'z'?)r|rrtzDelete failed: rNz Deleted 'z'.rr) rconfirmrdelete_inference_endpointrr/r0r1r2r3r4result)r#rr|rr;r/s r?deletersKK#D6,#6 5 !CP %%49E%R JJ4&#$/ P OE7+,##)C)C#D%OPsA B4"A B//B4zhf endpoints pause my-endpointc@t|} |j|||}tj|jy#t$rR}tjd|t j j|jj|d}~wwxYw)zPause an Inference Endpoint.rrtzPause failed: rN) rpause_inference_endpointrrr/r0r1r2r3r4r9r5rvs r?pausers 5 !CP//TYV[/\ HHX\\ P N5'*+##)C)C#D%OPrxzhf endpoints resume my-endpointfail_if_already_runningz--fail-if-already-runningzWIf `True`, the method will raise an error if the Inference Endpoint is already running.cDt|} |j|||| }tj|jy#t$rR}tjd|t j j|jj|d}~wwxYw)zResume an Inference Endpoint.r)r#rr running_okzResume failed: rN) rresume_inference_endpointrrr/r0r1r2r3r4r9r5)r#rrrr;r>r/s r?resumers 5 !C P0022 1 HHX\\ P OE7+,##)C)C#D%OPrkz&hf endpoints scale-to-zero my-endpointc@t|} |j|||}tj|jy#t$rR}tjd|t j j|jj|d}~wwxYw)z$Scale an Inference Endpoint to zero.rrtzScale To Zero failed: rN) r scale_to_zero_inference_endpointrrr/r0r1r2r3r4r9r5rvs r? scale_to_zeror s 5 !CP77TY^c7d HHX\\ P *5'23##)C)C#D%OPrx)NN)NNNN)N)NNNNNNNNNNNNNN)NFN)0__doc__r]typingrr0$huggingface_hub._inference_endpointsrrhuggingface_hub.errorsrr _cli_utilsr r r r r rrrrr _frameworkrr_outputrie_cli catalog_appstrNameArgNameOpt NamespaceOptcommandr@intfloatrArjro add_grouprwr{boolrrrrr[r?rs` 8  f;   ) F G!QR   "#%  $J  !#  $J n  '8:^&_`"&& & &a&DX)n(op\#      #' M|T |T  x   |T S  |T R  |T( a  )|T4 l  5|T@  g   A|TL  i   M|TZ[|T\  d V   ]|Th i|Tj i  k|Tv e  w|TB% d Y  C|TN&-4  O|TZ!  x  [|Tfg|Th d  y  i|Tv d p  w|TB  d ^   C|TN! d  !T   O|Tb d  U   c|Tv w|Txy|Tz{|T|!}|T~ d  !"78g  |TN O|Tq|T~ (.s-tu "  x     d c     !vF !  !  !L 0I/JKLY0N40>9-=>?#       @  KLM# Wd dd  d x   d d R  d d a  d* d l  +d6 d U  7dB d F  CdN  d V   OdZ d i  [df d e  gdr% d Y  sd~&-4  dJ!  x  KdV WdX YdNdN;<=# 0 00  w9: < 0 0 0>0,:;<#       =  ;<=#  '  'j    >6BCD#       E r