Hugging Face's logo

Buckets:
hf-doc-build
/
doc-dev

Files
xet
hf-doc-build/doc-dev / transformers.js /pr_1472 /en /api /utils /hub.md
rtrm's picture
rtrm
preview code
|
download
raw
18 kB

utils/hub

Utility functions to interact with the Hugging Face Hub (https://huggingface.co/models)

utils/hub.getFile(urlOrPath) ⇒ Promise.<(FileResponse|Response)>

Helper function to get a file, using either the Fetch API or FileSystem API.

Kind: static method of utils/hub
Returns: Promise.<(FileResponse|Response)> - A promise that resolves to a FileResponse object (if the file is retrieved using the FileSystem API), or a Response object (if the file is retrieved using the Fetch API). 

  ParamTypeDescription




urlOrPathURL | stringThe URL/path of the file to get.

utils/hub.getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path]) ⇒ Promise.<(string|Uint8Array)>

Retrieves a file from either a remote URL using the Fetch API or from the local file system using the FileSystem API. If the filesystem is available and env.useCache = true, the file will be downloaded and cached.

Kind: static method of utils/hub
Returns: Promise.<(string|Uint8Array)> - A Promise that resolves with the file content as a Uint8Array if return_path is false, or the file path as a string if return_path is true.
Throws:

  • Will throw an error if the file is not found and fatal is true.

    ParamTypeDefaultDescription

    path_or_repo_idstringThis can be either:

a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.

filenamestringThe name of the file to locate in path_or_repo.


[fatal]booleantrueWhether to throw an error if the file is not found.


[options]PretrainedOptionsAn object containing optional parameters.


[return_path]booleanfalseWhether to return the path of the file instead of the file content.

getModelFile~cacheKey : string

Kind: inner property of getModelFile

getModelFile~response : Response | FileResponse | undefined

Kind: inner property of getModelFile

getModelFile~buffer : Uint8Array

Kind: inner property of getModelFile

utils/hub.getModelText(modelPath, fileName, [fatal], [options]) ⇒ Promise.<(string|null)>

Fetches a text file from a given path and file name.

Kind: static method of utils/hub
Returns: Promise.<(string|null)> - The text content of the file.
Throws:

  • Will throw an error if the file is not found and fatal is true.

    ParamTypeDefaultDescription

    modelPathstringThe path to the directory containing the file.

    fileNamestringThe name of the file to fetch.

    [fatal]booleantrueWhether to throw an error if the file is not found.

    [options]PretrainedOptionsAn object containing optional parameters.

utils/hub.getModelJSON(modelPath, fileName, [fatal], [options]) ⇒ Promise.<Object>

Fetches a JSON file from a given path and file name.

Kind: static method of utils/hub
Returns: Promise.<Object> - The JSON data parsed into a JavaScript object.
Throws:

  • Will throw an error if the file is not found and fatal is true.

    ParamTypeDefaultDescription

    modelPathstringThe path to the directory containing the file.

    fileNamestringThe name of the file to fetch.

    [fatal]booleantrueWhether to throw an error if the file is not found.

    [options]PretrainedOptionsAn object containing optional parameters.

utils/hub~FileResponse

Kind: inner class of utils/hub

new FileResponse(filePath)

Creates a new FileResponse object.

  ParamType




filePathstring

fileResponse.updateContentType() ⇒ void

Updates the 'content-type' header property of the response based on the extension of the file specified by the filePath property of the current object.

Kind: instance method of FileResponse

fileResponse.clone() ⇒ FileResponse

Clone the current FileResponse object.

Kind: instance method of FileResponse
Returns: FileResponse - A new FileResponse object with the same properties as the current object.

fileResponse.arrayBuffer() ⇒ Promise.<ArrayBuffer>

Reads the contents of the file specified by the filePath property and returns a Promise that resolves with an ArrayBuffer containing the file's contents.

Kind: instance method of FileResponse
Returns: Promise.<ArrayBuffer> - A Promise that resolves with an ArrayBuffer containing the file's contents.
Throws:

  • Error If the file cannot be read.

fileResponse.blob() ⇒ Promise.<Blob>

Reads the contents of the file specified by the filePath property and returns a Promise that resolves with a Blob containing the file's contents.

Kind: instance method of FileResponse
Returns: Promise.<Blob> - A Promise that resolves with a Blob containing the file's contents.
Throws:

  • Error If the file cannot be read.

fileResponse.text() ⇒ Promise.<string>

Reads the contents of the file specified by the filePath property and returns a Promise that resolves with a string containing the file's contents.

Kind: instance method of FileResponse
Returns: Promise.<string> - A Promise that resolves with a string containing the file's contents.
Throws:

  • Error If the file cannot be read.

fileResponse.json() ⇒ Promise.<Object>

Reads the contents of the file specified by the filePath property and returns a Promise that resolves with a parsed JavaScript object containing the file's contents.

Kind: instance method of FileResponse
Returns: Promise.<Object> - A Promise that resolves with a parsed JavaScript object containing the file's contents.
Throws:

  • Error If the file cannot be read.

utils/hub~FileCache

Kind: inner class of utils/hub

new FileCache(path)

Instantiate a FileCache object.

  ParamType




pathstring

fileCache.match(request) ⇒ Promise.<(FileResponse|undefined)>

Checks whether the given request is in the cache.

Kind: instance method of FileCache 

  ParamType




requeststring

fileCache.put(request, response, [progress_callback]) ⇒ Promise.<void>

Adds the given response to the cache.

Kind: instance method of FileCache 

  ParamTypeDescription




requeststring

responseResponse

[progress_callback]*Optional.

The function to call with progress updates

utils/hub~CONTENT_TYPE_MAP

Mapping from file extensions to MIME types.

Kind: inner constant of utils/hub

utils/hub~isValidUrl(string, [protocols], [validHosts]) ⇒ boolean

Determines whether the given string is a valid URL.

Kind: inner method of utils/hub
Returns: boolean - True if the string is a valid URL, false otherwise. 

  ParamTypeDefaultDescription




stringstring | URLThe string to test for validity as an URL.


[protocols]Array.&lt;string&gt;A list of valid protocols. If specified, the protocol must be in this list.


[validHosts]Array.&lt;string&gt;A list of valid hostnames. If specified, the URL&#39;s hostname must be in this list.

utils/hub~isValidHfModelId(string) ⇒ boolean

Tests whether a string is a valid Hugging Face model ID or not. Adapted from https://github.com/huggingface/huggingface_hub/blob/6378820ebb03f071988a96c7f3268f5bdf8f9449/src/huggingface_hub/utils/_validators.py#L119-L170

Kind: inner method of utils/hub
Returns: boolean - True if the string is a valid model ID, false otherwise. 

  ParamTypeDescription




stringstringThe string to test

utils/hub~handleError(status, remoteURL, fatal) ⇒ null

Helper method to handle fatal errors that occur while trying to load a file from the Hugging Face Hub.

Kind: inner method of utils/hub
Returns: null - Returns null if fatal = true.
Throws:

  • Error If fatal = false.

    ParamTypeDescription

    statusnumberThe HTTP status code of the error.

    remoteURLstringThe URL of the file that could not be loaded.

    fatalbooleanWhether to raise an error if the file could not be loaded.

utils/hub~tryCache(cache, ...names) ⇒ Promise.<(FileResponse|Response|undefined)>

Kind: inner method of utils/hub
Returns: Promise.<(FileResponse|Response|undefined)> - The item from the cache, or undefined if not found. 

  ParamTypeDescription




cacheFileCache | CacheThe cache to search


...namesArray.&lt;string&gt;The names of the item to search for

utils/hub~readResponse(response, progress_callback) ⇒ Promise.<Uint8Array>

Read and track progress when reading a Response object

Kind: inner method of utils/hub
Returns: Promise.<Uint8Array> - A Promise that resolves with the Uint8Array buffer 

  ParamTypeDescription




responseResponse | FileResponseThe Response object to read


progress_callback*The function to call with progress updates

utils/hub~pathJoin(...parts) ⇒ string

Joins multiple parts of a path into a single path, while handling leading and trailing slashes.

Kind: inner method of utils/hub
Returns: string - A string representing the joined path. 

  ParamTypeDescription




...partsstringMultiple parts of a path.

utils/hub~ExternalData : boolean | number

Whether to load the model using the external data format (used for models >= 2GB in size). If true, the model will be loaded using the external data format. If a number, this many chunks will be loaded using the external data format (of the form: "model.onnx_data[_{chunk_number}]").

Kind: inner typedef of utils/hub

utils/hub~PretrainedOptions : Object

Options for loading a pretrained model.

Kind: inner typedef of utils/hub
Properties

  NameTypeDefaultDescription




[progress_callback]*If specified, this function will be called during model construction, to provide the user with progress updates.


[config]*Configuration for the model to use instead of an automatically loaded configuration. Configuration can be automatically loaded when:

The model is a model provided by the library (loaded with the model id string of a pretrained model). The model is loaded by supplying a local directory as pretrained_model_name_or_path and a configuration JSON file named config.json is found in the directory.

[cache_dir]stringnullPath to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.


[local_files_only]booleanfalseWhether or not to only look at local files (e.g., not try downloading the model).


[revision]string&quot;&#x27;main&#x27;&quot;The specific model version to use. It can be a branch name, a tag name, or a commit id,

since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git. NOTE: This setting is ignored for local requests.

utils/hub~ModelSpecificPretrainedOptions : Object

Options for loading a pretrained model.

Kind: inner typedef of utils/hub
Properties

  NameTypeDefaultDescription




[subfolder]string&quot;&#x27;onnx&#x27;&quot;In case the relevant files are located inside a subfolder of the model repo on huggingface.co,

you can specify the folder name here.

[model_file_name]stringnullIf specified, load the model with this name (excluding the .onnx suffix). Currently only valid for encoder- or decoder-only models.


[device]*The device to run the model on. If not specified, the device will be chosen from the environment settings.


[dtype]*The data type to use for the model. If not specified, the data type will be chosen from the environment settings.


[use_external_data_format]ExternalData | Record&lt;string, ExternalData&gt;falseWhether to load the model using the external data format (used for models &gt;= 2GB in size).


[session_options]*(Optional) User-specified session options passed to the runtime. If not provided, suitable defaults will be chosen.

utils/hub~PretrainedModelOptions : *

Options for loading a pretrained model.

Kind: inner typedef of utils/hub

Xet Storage Details

Size:
18 kB
·
Xet hash:
6f0dfba50dfbe1990b86fd0711ea10bc9d5a1acc604741b50dc8cf1dcc427228

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.