^jv,ddlZddlmZddlmZmZddlZddlmZ ddl m Z ddl m Z ddlmZmZmZmZdd lmZmZmZmZmZmZmZmZdd lmZmZmZm Z dd l!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'dd l(m)Z)m*Z*m+Z+m,Z,dd l-m.Z.m/Z/m0Z0m1Z1m2Z2ddl3m4Z4ddl3mZ5e+jle7Z8eddddddde jrdddddddddddde:de:dzde:dzde:ezdzde:ezdzde:dzde:dzde;e:zdzdee:e:zdzde>e:e:zdzd e?d!e@e dzd"e;e:e:fdzd#e:dzd$edd%e:f(d&ZAeddddddde jrdddddddddd'dde:de:dzde:dzde:ezdzde:ezdzde:dzde:dzde;e:zdzdee:e:zdzde>e:e:zdzd e?d!e@e dzd"e;e:e:fdzd#e:dzd$ed'd%e>ef(d(ZAeddddddde jrdddddddddddde:de:dzde:dzde:ezdzde:ezdzde:dzde:dzde;e:zdzdee:e:zdzde>e:e:zdzd e?d!e@e dzd"e;e:e:fdzd#e:dzd$e=d%e:e>ezf(d)ZAe,ddddddde jrdddddddddddde:de:dzde:dzde:ezdzde:ezdzde:dzde:dzde;e:zdzdee:e:zdzde>e:e:zdzd e?d!e@e dzd"e;e:e:fdzd#e:dzd$e=d%e:e>ezf(d*ZAd+e:d,e:d-e:de>e:e:zdzde>e:e:zdzde:de:d.eBdzd%dfd/ZCe,ddddd0de:de:dzde:dzde:ezdzde:ezdzd%e>e&f d1ZDd-e:d2e:d%e=fd3ZEy)4N)Path)Literaloverload)tqdm) thread_map) constants)TreeCacheEntryread_tree_cachetree_cache_folder_for_local_dirwrite_tree_cache)CachedRepoTreeNotFoundError DryRunErrorGatedRepoErrorHfHubHTTPErrorIncompleteSnapshotErrorLocalEntryNotFoundErrorRepositoryNotFoundErrorRevisionNotFoundError)REGEX_COMMIT_HASHDryRunFileInfohf_hub_downloadrepo_folder_name) DatasetInfoHfApi KernelInfo ModelInfoRepoFile SpaceInfo)OfflineModeIsEnabledfilter_repo_objectsloggingvalidate_hf_hub_args)XET_BYTES_BAR_FORMATXET_TRANSFER_BAR_FORMAT_finish_transfer_bar_set_aggregate_rate_postfix_update_transfer_bar)_create_progress_barF) repo_typerevision cache_dir local_dir library_namelibrary_version user_agent etag_timeoutforce_downloadtokenlocal_files_onlyallow_patternsignore_patterns max_workers tqdm_classheadersendpointdry_runrepo_idr+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<returncyNr=r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<s m/var/www/ramen.bs-engineer-server.com/venv/lib/python3.12/site-packages/huggingface_hub/_snapshot_download.pysnapshot_downloadrD&s, Tcyr@rArBs rCrDrD?s,rEcyr@rArBs rCrDrDXs,"%rEc  !"#$%tjttj j 0ttj j |tj }dtjvr(tddttjtjjt} tn|}t }d}d}| s |j|}|r t/d|d"t1j2|r|"ndtjj|d |}tjj5|r$t7|5}|j9"ddd"WUtjj|d "}tjj5|rt;|"|| | || |Stj=rlt?jArS"t;|"t| | || tBjEd d |dtj S| r tGdtI|t(r tGd|tI|tJtLfs)tI|t,r|jNjPdk(r|tGd|jRjTd|d||jVJd|jV"tY|"}||j[d"Dcic]}tI|t\r}|jt_|j`|jb|jd|jdjfnd|jd|jdj`nd|jh}}s tk|"|tmto|jq| | }dts|d}rd|z}tjj|d "}|"k7r|tjj|d |} tjttjjw|dt7|d5}|jy"dddg$|xst|}t|tBjd!d"d#d#d$dt% %t|tBjd&d'd#d#d$dt% #G#%fd(d)!d*td+df!" $ fd, } t| ||||-t%%jd.#jd/rtd0$DsJ$S(ttjjS|S#t j"$rt j$t j&t(f$r }|}Yd}~d}~wt*$rt,$r }|}Yd}~3d}~wwxYw#1swYxYwcc}w#1swYxYw#tz$r'}tBjEd|d|d Yd}~d}~wwxYw)1aDownload repo files. Download a whole snapshot of a repo's files at the specified revision. This is useful when you want all files from a repo because you don't know which ones you will need _a priori_. All files are nested in a folder to keep their path and filename relative to that folder. You can also filter which files to download by using `allow_patterns` and `ignore_patterns`. If `local_dir` is provided, the file structure from the repo will be replicated in this location. When using this option, the `cache_dir` will not be used, and a `.cache/huggingface/` folder will be created at the root of `local_dir` to store some metadata related to the downloaded files. While this mechanism is not as robust as the main cache system, it's optimized for regularly pulling the latest version of a repository. An alternative would be to clone the repo, but this requires git and git-lfs to be installed and properly configured. It is also not possible to filter which files to download when cloning a repository using git. Args: repo_id (`str`): A user or an organization name and a repo name separated by a `/`. repo_type (`str`, *optional*): Set to `"dataset"`, `"space"` or `"kernel"` if downloading from a dataset, space or kernel repo, `None` or `"model"` if downloading from a model. Default is `None`. revision (`str`, *optional*): An optional Git revision id, which can be a branch name, a tag, or a commit hash. cache_dir (`str`, `Path`, *optional*): Path to the folder where cached files are stored. local_dir (`str` or `Path`, *optional*): If provided, the downloaded files will be placed under this directory. library_name (`str`, *optional*): The name of the library to which the object corresponds. library_version (`str`, *optional*): The version of the library. user_agent (`str`, `dict`, *optional*): The user-agent info in the form of a dictionary or a string. etag_timeout (`float`, *optional*, defaults to `10`): When fetching ETag, how many seconds to wait for the server to send data before giving up, which is passed to `httpx.request`. force_download (`bool`, *optional*, defaults to `False`): Whether the file should be downloaded even if it already exists in the local cache. token (`str`, `bool`, *optional*): A token to be used for the download. - If `True`, the token is read from the HuggingFace config folder. - If a string, it's used as the authentication token. headers (`dict`, *optional*): Additional headers to include in the request. Those headers take precedence over the others. endpoint (`str`, *optional*): The Hub endpoint to send the request to. Defaults to the value of `HF_ENDPOINT`. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, do not download any files even if they are not in `cache_dir` or `local_dir`. allow_patterns (`list[str]` or `str`, *optional*): If provided, only files matching at least one pattern are downloaded. ignore_patterns (`list[str]` or `str`, *optional*): If provided, files matching any of the patterns are not downloaded. max_workers (`int`, *optional*): Number of concurrent threads to download files (1 thread = 1 file download). Defaults to 8. tqdm_class (`tqdm`, *optional*): If provided, overwrites the default behavior for the progress bar. Passed argument must inherit from `tqdm.auto.tqdm` or at least mimic its behavior. Note that the `tqdm_class` is not passed to each individual download. Defaults to the custom HF progress bar that can be disabled by setting `HF_HUB_DISABLE_PROGRESS_BARS` environment variable. dry_run (`bool`, *optional*, defaults to `False`): If `True`, perform a dry run without actually downloading the files. Returns a list of [`DryRunFileInfo`] objects containing information about what would be downloaded. Returns: `str` or list of [`DryRunFileInfo`]: - If `dry_run=False`: Local snapshot path. - If `dry_run=True`: A list of [`DryRunFileInfo`] objects containing download information. Raises: [`~utils.RepositoryNotFoundError`] If the repository to download from cannot be found. This may be because it doesn't exist or because it is set to `private` and you do not have access. [`~utils.RevisionNotFoundError`] If the revision to download from cannot be found. [`~errors.IncompleteSnapshotError`] If the Hub cannot be reached (offline, connection issue, or `local_files_only=True`) and the cached snapshot is missing some of the requested files. [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) If `token=True` and the token cannot be found. [`OSError`](https://docs.python.org/3/library/exceptions.html#OSError) if ETag cannot be determined. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If some parameter value is invalid. NmodelInvalid repo type: . Accepted repo types are: r=r+)r/r0r1r;r:r4)r=r+r,zDry run cannot be performed as the repository cannot be accessed. Please check your internet connection or authentication token.refs snapshots)tree_cache_folder commit_hashbase_dirr6r7r=r,api_call_errorzReturning existing local_dir `z<` as remote repo cannot be accessed in `snapshot_download` ().zCannot find an appropriate cached snapshot folder for the specified revision on the local disk and outgoing traffic has been disabled. To enable repo look-ups and downloads online, pass 'local_files_only=False' as input.zCannot find an appropriate cached snapshot folder for the specified revision on the local disk and outgoing traffic has been disabled. To enable repo look-ups and downloads online, set 'HF_HUB_OFFLINE=0' as environment variable.izGot: : z An error happened while trying to locate the files on the Hub, and we cannot find the appropriate snapshot folder for the specified revision on the local disk. Please check your internet connection and try again.z8Repo info returned from server must have a revision sha.T)r= recursiver,r+)sizeblob_id lfs_sha256lfs_sizexet_hashitemsr6r7z Fetching z filesz [dry-run] )exist_okwz+Ignored error while writing commit hash to .z*huggingface_hub.snapshot_download.transferzDownloading bytesrB) cls log_levelnamedesctotalinitialunit unit_scale bar_formatz!huggingface_hub.snapshot_downloadz$Reconstructing (incomplete total...)ceZdZdZfdZdZdZddZddee zdzddffd Z ddee zdzddffd Z dd e d e ddffd Zdd e d e ddffd Zy)*snapshot_download.._AggregatedTqdmaFake tqdm object to aggregate progress into the parent snapshot progress bars. In practice, the `_AggregatedTqdm` object won't be displayed; it's just used to update the `reconstruct_progress` and `transfer_progress` bars from each thread/file download. c|jdd}|@jxsd|z_jxsd|z_j|jdd}|rj|yy)Nrerrf)poprerefreshupdate)selfargskwargsrerfreconstruct_progresstransfer_progresss rC__init__z3snapshot_download.._AggregatedTqdm.__init__sJJw-E .B.H.H.MAQV-V$*+<+B+B+Ga5*P!'$,,.jjA.G$++G4rEc|Sr@rArps rC __enter__z4snapshot_download.._AggregatedTqdm.__enter__sKrEcyr@rA)rpexc_type exc_value tracebacks rC__exit__z3snapshot_download.._AggregatedTqdm.__exit__ rEr>Ncyr@rArws rCclosez0snapshot_download.._AggregatedTqdm.closer~rEnc(j|yr@)ro)rprrss rCroz1snapshot_download.._AggregatedTqdm.updates ' ' *rEc8tt|xsdy)Nr)r(int)rprrts rCupdate_transferz:snapshot_download.._AggregatedTqdm.update_transfers !2CQK @rEpostfixrnctyr@r')rprrnrss rCset_postfix_strz:snapshot_download.._AggregatedTqdm.set_postfix_strs '(< =rEctyr@r)rprrnrts rCset_transfer_postfix_strzCsnapshot_download.._AggregatedTqdm.set_transfer_postfix_strs '(9 :rE)r>N)r)F)__name__ __module__ __qualname____doc__rurxr}rrfloatrorstrboolrr)rsrtsrC_AggregatedTqdmrks 5    +C%K$. +t + AS5[4%7 A A >3 > >$ > ;C ;$ ;SW ;rEr repo_filer>cXjt f|  dy)N)filenamer+r,r;r-r.r/r0r1r2r3r4r:r9r<)appendr)rrr-rPr<r;r2r3r:r/r0r.r=r+resultsr4r1s rC_inner_hf_hub_downloadz1snapshot_download.._inner_hf_hub_downloadsL  "#$!##) /%)-*!  rE)rdr8r9zDownload completezReconstruction completec3<K|]}t|tywr@) isinstancer).0rs rC z$snapshot_download..sBQ:a0Bs)Hr HF_HUB_CACHErr expanduserresolveDEFAULT_REVISIONREPO_TYPES_WITH_KERNEL ValueErrorospathjoinrr r repo_infohttpx ProxyError ConnectErrorTimeoutExceptionr rrrrmatchexistsopenread_raise_if_incomplete_snapshotis_diranyiterdirloggerwarningrrrrresponse status_code __class__rshar list_repo_treerr rVrWlfssha256rZr listr!keyslenmakedirsdirnamewriteOSErrorhf_tqdmr)getEffectiveLevelr%r$rr&set_descriptionallrealpath)&r=r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<storage_folderrOapirrRerrorref_pathfsnapshot_folder tree_entriesfiltered_repo_files tqdm_descerrrPrsrrts&`` ```````` ``` @@@@@rCrDrDqs^** DO..088:;IY224<<>? --  888!),GILlLlHmGn o  WW\\)-=gYb-cdNGPF[7 Bao !'  CJNI'+N   gU] ^I> S! "   " "8 ,"Kww||NFHEHww~~h'(^+q"#&&(K+  "y'8 ggll>; TOww~~o..&7 +,#1$3#%#1 '&  YI!c)*;*;*=&>*1*;$/!$Y'5(7 '!)'5 4YK?{}K|LLNO9,,.// )5  (< =)>"  " )@.(Q R ~~ 6>;R;R;^;^be;e! *0099:"^; LO;77<<A [ KK1D Ah$ % $ % +-G&wJ- **, 9  * 0 **, 0 3 ' (;(;Z # $   ,  *+%%&9:(()BCB'BBBB277##I.// c  ""E$:$:.json` cache files) and is used to skip network calls on subsequent downloads. This function exposes that cache directly. If you need the current tree listing of a repo on the Hub, use [`list_repo_tree`] instead. Args: repo_id (`str`): A user or an organization name and a repo name separated by a `/`. repo_type (`str`, *optional*): Set to `"dataset"`, `"space"` or `"kernel"` if listing from a dataset, space or kernel repo, `None` or `"model"` if listing from a model. Default is `None`. revision (`str`, *optional*): An optional Git revision id, which can be a branch name, a tag, or a commit hash. Defaults to the default branch. Branch/tag names are resolved to a commit hash using the local cache (`refs/`). cache_dir (`str`, `Path`, *optional*): Path to the folder where cached files are stored. Defaults to the value of `HF_HUB_CACHE`. local_dir (`str` or `Path`, *optional*): If provided, read the tree listing cached by a `local_dir` download (from `local_dir/.cache/huggingface/`) instead of the main cache. Branch/tag revisions are still resolved to a commit hash using the main cache (`cache_dir`). Returns: `list[RepoFile]`: The list of [`RepoFile`] objects cached for this revision. Raises: [`~errors.CachedRepoTreeNotFoundError`] If no tree listing is cached for the requested revision (e.g. the repo was never downloaded at this revision). Example: ```py >>> from huggingface_hub import get_cached_repo_tree >>> files = get_cached_repo_tree("openai-community/gpt2") >>> [f.path for f in files] ['.gitattributes', 'config.json', 'model.safetensors', ...] ``` NrJrKrLrMz"No cached tree listing found for 'rz', repo_type 'zH'): the revision is not a commit hash and no matching ref is cached in 'zQ'. Download the repo (e.g. with `snapshot_download`) to populate the cache first.z ', commit 'z') in ')rrVoidxetHash)r rrrrrrREPO_TYPE_MODELrrrrrrr rrisfilerrrr r\rrVrWrZ) r=r+r,r-r.rrOrPrrrrentrys rCget_cached_repo_treerAs*`** DO..088:;IY224<<>? ----  888!),GILlLlHmGn o  WW\\)-=gYb-cdNGPF[7 Baox( 77<<Aww~~h'-4WI]8*TbclbmnWWeVfgaa  (^ #q&&(K ###4kBL)0 xjP[\g[hi#G,=+>?+ +  (--/  D% dW  # # sH7H$H!rctjj|g|jd}tjdk(r^t tjj |dkDr3|jds"dtjj |z}tjj|S)zCheck whether a repo file (path relative to `base_dir`, '/'-separated) exists on disk. On Windows, paths longer than 255 characters must be prefixed with `\\?\`, otherwise `os.path.isfile` reports an existing file as missing. /ntz\\?\) rrrsplitrcrabspath startswithr)rQr full_paths rCrrs  X8 38I ww$3rwwy9:S@I]I]^gIh :: 77>>) $$rE)Frpathlibrtypingrrr tqdm.autor base_tqdmtqdm.contrib.concurrentrr _tree_cacher r r r errorsrrrrrrrr file_downloadrrrrhf_apirrrrrrutilsr r!r"r#utils._xet_progress_reportingr$r%r&r'r( utils.tqdmr)r get_loggerrrDEFAULT_ETAG_TIMEOUTrdictrrrrtyperD ExceptionrrrrArErCrs $ '.kk   `_RR[[-'   H % !#'#'#"&$(#88 #"-1.2)-%)#) TzDj  TzD  TzD *4Zs T! #: IOd*#Y_t+ !"Y$&#$#s(^d "%&Dj'(U^)* + 0 !#'#'#"&$(#88 #"-1.2)-%)!) TzDj  TzD  TzD *4Zs T! #: IOd*#Y_t+ !"Y$&#$#s(^d "%&Dj'(T])* .+ 0 !#'#'#"&$(#88 #"-1.2)-%))% %Tz%Dj % TzD % TzD %*%4Z%s T!%%% #: %%IOd*%#Y_t+% !%"Y$&#%$#s(^d "%%&Dj'%()%* 4 +% %0!#'#'#"&$(#88 #"-1.2)-%))c cTzcDj c TzD c TzD c*c4Zcs T!ccc #: ccIOd*c#Y_t+c !c"Y$&#c$#s(^d "%c&Dj'c()c* 4 +ccL &&& & IOd* & #Y_t+ &&&$& &R!#'#' ] ]Tz]Dj ] TzD ] TzD ] (^]]@ % %C %D %rE