transformers.js/pr_1649/en/api/utils/hub.md

utils/hub

Utility functions to interact with the Hugging Face Hub (https://huggingface.co/models)

• utils/hub
  • static
    • .getFile(urlOrPath) ⇒ Promise.<(FileResponse|Response)>
    • .getFetchHeaders(urlOrPath) ⇒ Headers
    • .buildResourcePaths(path_or_repo_id, filename, [options], [cache]) ⇒ Object
    • .checkCachedResource(cache, localPath, proposedCacheKey) ⇒ Promise.<(Response|FileResponse|undefined|string)>
    • .storeCachedResource(path_or_repo_id, filename, cache, cacheKey, response, [result], [options]) ⇒ Promise.<void>
    • .loadResourceFile(path_or_repo_id, filename, [fatal], [options], [return_path], [cache]) ⇒ Promise.<(string|Uint8Array|null)>
      • ~cacheKey : string
      • ~response : Response | FileResponse | undefined | string
      • ~buffer : Uint8Array
    • .getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path]) ⇒ Promise.<(string|Uint8Array)>
      • ~cache : CacheInterface | null
    • .getModelText(modelPath, fileName, [fatal], [options]) ⇒ Promise.<(string|null)>
    • .getModelJSON(modelPath, fileName, [fatal], [options]) ⇒ Promise.<Object>
  • inner
    • ~INFLIGHT_LOADS : Map.<string, Promise.<(string|Uint8Array)>>
    • ~ExternalData : boolean | number
    • ~PretrainedOptions : Object
    • ~ModelSpecificPretrainedOptions : Object
    • ~PretrainedModelOptions : PretrainedOptions | ModelSpecificPretrainedOptions

---

utils/hub.getFile(urlOrPath) ⇒ Promise.<(FileResponse|Response)>

Helper function to get a file, using either the Fetch API or FileSystem API.

Kind: static method of utils/hub
Returns: Promise.<(FileResponse|Response)> - A promise that resolves to a FileResponse object (if the file is retrieved using the FileSystem API), or a Response object (if the file is retrieved using the Fetch API).

  ParamTypeDescription




urlOrPathURL | stringThe URL/path of the file to get.

  

---

utils/hub.getFetchHeaders(urlOrPath) ⇒ Headers

Generates appropriate HTTP headers for fetching resources. In Node.js environments, adds User-Agent and Authorization headers when applicable. In browser environments, returns minimal headers for security.

Kind: static method of utils/hub
Returns: Headers - A Headers object with appropriate headers for the request.

  ParamTypeDescription




urlOrPathURL | stringThe URL or path being fetched.

  

---

utils/hub.buildResourcePaths(path_or_repo_id, filename, [options], [cache]) ⇒ Object

Builds the resource paths and URLs for a model file. Can be used to get the resource URL or path without loading the file.

Kind: static method of utils/hub
Returns: Object - An object containing all the paths and URLs for the resource.

  ParamTypeDescription




path_or_repo_idstringThis can be either:

a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.

filenamestringThe name of the file to locate.


[options]PretrainedOptionsAn object containing optional parameters.


[cache]CacheInterface | nullThe cache instance to use for determining cache keys.

  

---

utils/hub.checkCachedResource(cache, localPath, proposedCacheKey) ⇒ Promise.<(Response|FileResponse|undefined|string)>

Checks if a resource exists in cache.

Kind: static method of utils/hub
Returns: Promise.<(Response|FileResponse|undefined|string)> - The cached response if found, undefined otherwise.

  ParamTypeDescription




cacheCacheInterface | nullThe cache instance to check.


localPathstringThe local path to try first.


proposedCacheKeystringThe proposed cache key to try second.

  

---

utils/hub.storeCachedResource(path_or_repo_id, filename, cache, cacheKey, response, [result], [options]) ⇒ Promise.<void>

Stores a resource in the cache.

Kind: static method of utils/hub

  ParamTypeDescription




path_or_repo_idstringThe path or repo ID of the model.


filenamestringThe name of the file to cache.


cacheCacheInterfaceThe cache instance to store in.


cacheKeystringThe cache key to use.


responseResponse | FileResponseThe response to cache.


[result]Uint8ArrayThe result buffer if already read.


[options]PretrainedOptionsOptions containing progress callback and context for progress updates.

  

---

utils/hub.loadResourceFile(path_or_repo_id, filename, [fatal], [options], [return_path], [cache]) ⇒ Promise.<(string|Uint8Array|null)>

Loads a resource file from local or remote sources.

Kind: static method of utils/hub
Returns: Promise.<(string|Uint8Array|null)> - A Promise that resolves with the file content as a Uint8Array if return_path is false, or the file path as a string if return_path is true.
Throws:

• Will throw an error if the file is not found and fatal is true.

  ParamTypeDefaultDescription
  

  path_or_repo_idstringThis can be either:

a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.

filenamestringThe name of the file to locate.


[fatal]booleantrueWhether to throw an error if the file is not found.


[options]PretrainedOptionsAn object containing optional parameters.


[return_path]booleanfalseWhether to return the path of the file instead of the file content.


[cache]CacheInterface | nullThe cache instance to use.

  

• .loadResourceFile(path_or_repo_id, filename, [fatal], [options], [return_path], [cache]) ⇒ Promise.<(string|Uint8Array|null)>
  • ~cacheKey : string
  • ~response : Response | FileResponse | undefined | string
  • ~buffer : Uint8Array

---

loadResourceFile~cacheKey : string

Kind: inner property of loadResourceFile

---

loadResourceFile~response : Response | FileResponse | undefined | string

Kind: inner property of loadResourceFile

---

loadResourceFile~buffer : Uint8Array

Kind: inner property of loadResourceFile

---

utils/hub.getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path]) ⇒ Promise.<(string|Uint8Array)>

Retrieves a file from either a remote URL using the Fetch API or from the local file system using the FileSystem API. If the filesystem is available and env.useCache = true, the file will be downloaded and cached.

Kind: static method of utils/hub
Returns: Promise.<(string|Uint8Array)> - A Promise that resolves with the file content as a Uint8Array if return_path is false, or the file path as a string if return_path is true.
Throws:

• Will throw an error if the file is not found and fatal is true.

  ParamTypeDefaultDescription
  

  path_or_repo_idstringThis can be either:

a string, the model id of a model repo on huggingface.co. a path to a directory potentially containing the file.

filenamestringThe name of the file to locate in path_or_repo.


[fatal]booleantrueWhether to throw an error if the file is not found.


[options]PretrainedOptionsAn object containing optional parameters.


[return_path]booleanfalseWhether to return the path of the file instead of the file content.

  

---

getModelFile~cache : CacheInterface | null

Kind: inner constant of getModelFile

---

utils/hub.getModelText(modelPath, fileName, [fatal], [options]) ⇒ Promise.<(string|null)>

Fetches a text file from a given path and file name.

Kind: static method of utils/hub
Returns: Promise.<(string|null)> - The text content of the file.
Throws:

• Will throw an error if the file is not found and fatal is true.

  ParamTypeDefaultDescription
  

  modelPathstringThe path to the directory containing the file.

  fileNamestringThe name of the file to fetch.

  [fatal]booleantrueWhether to throw an error if the file is not found.

  [options]PretrainedOptionsAn object containing optional parameters.

---

utils/hub.getModelJSON(modelPath, fileName, [fatal], [options]) ⇒ Promise.<Object>

Fetches a JSON file from a given path and file name.

Kind: static method of utils/hub
Returns: Promise.<Object> - The JSON data parsed into a JavaScript object.
Throws:

• Will throw an error if the file is not found and fatal is true.

  ParamTypeDefaultDescription
  

  modelPathstringThe path to the directory containing the file.

  fileNamestringThe name of the file to fetch.

  [fatal]booleantrueWhether to throw an error if the file is not found.

  [options]PretrainedOptionsAn object containing optional parameters.

---

utils/hub~INFLIGHT_LOADS : Map.<string, Promise.<(string|Uint8Array)>>

In-flight file loads keyed by repo+filename.

Kind: inner constant of utils/hub

---

utils/hub~ExternalData : boolean | number

Specifies whether to load the model using the external data format.

• false: Do not use external data format
• true: Use external data format with 1 chunk
• number: Use external data format with the specified number of chunks

Kind: inner typedef of utils/hub

---

utils/hub~PretrainedOptions : Object

Options for loading a pretrained model.

Kind: inner typedef of utils/hub
Properties

  NameTypeDefaultDescription




[progress_callback]ProgressCallbackIf specified, this function will be called during model construction, to provide the user with progress updates.


[config]PretrainedConfigConfiguration for the model to use instead of an automatically loaded configuration. Configuration can be automatically loaded when:

The model is a model provided by the library (loaded with the model id string of a pretrained model). The model is loaded by supplying a local directory as pretrained_model_name_or_path and a configuration JSON file named config.json is found in the directory.

[cache_dir]stringnullPath to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.


[local_files_only]booleanfalseWhether or not to only look at local files (e.g., not try downloading the model).


[revision]string"'main'"The specific model version to use. It can be a branch name, a tag name, or a commit id,

since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git. NOTE: This setting is ignored for local requests.

---

utils/hub~ModelSpecificPretrainedOptions : Object

Options for loading a pretrained model.

Kind: inner typedef of utils/hub
Properties

  NameTypeDefaultDescription




[subfolder]string"'onnx'"In case the relevant files are located inside a subfolder of the model repo on huggingface.co,

you can specify the folder name here.

[model_file_name]stringnullIf specified, load the model with this name (excluding the dtype and .onnx suffixes). Currently only valid for encoder- or decoder-only models.


[device]DeviceType | Record.<string, DeviceType>The device to run the model on. If not specified, the device will be chosen from the environment settings.


[dtype]DataType | Record.<string, DataType>The data type to use for the model. If not specified, the data type will be chosen from the environment settings.


[use_external_data_format]ExternalData | Record.<string, ExternalData>falseWhether to load the model using the external data format (used for models >= 2GB in size).


[session_options]InferenceSession.SessionOptions(Optional) User-specified session options passed to the runtime. If not provided, suitable defaults will be chosen.

  

---

utils/hub~PretrainedModelOptions : PretrainedOptions | ModelSpecificPretrainedOptions

Options for loading a pretrained model.

Kind: inner typedef of utils/hub

---