transformers.js/pr_1472/en/api/utils/hub.md

# utils/hub

Utility functions to interact with the Hugging Face Hub (https://huggingface.co/models)

* [utils/hub](#module_utils/hub)
    * _static_
        * [`.getFile(urlOrPath)`](#module_utils/hub.getFile) ⇒ Promise.<(FileResponse|Response)>
        * [`.getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path])`](#module_utils/hub.getModelFile) ⇒ Promise.<(string|Uint8Array)>
            * [`~cacheKey`](#module_utils/hub.getModelFile..cacheKey) : string
            * [`~response`](#module_utils/hub.getModelFile..response) : Response | FileResponse | undefined
            * [`~buffer`](#module_utils/hub.getModelFile..buffer) : Uint8Array
        * [`.getModelText(modelPath, fileName, [fatal], [options])`](#module_utils/hub.getModelText) ⇒ Promise.<(string|null)>
        * [`.getModelJSON(modelPath, fileName, [fatal], [options])`](#module_utils/hub.getModelJSON) ⇒ Promise.<Object>
    * _inner_
        * [~FileResponse](#module_utils/hub..FileResponse)
            * [`new FileResponse(filePath)`](#new_module_utils/hub..FileResponse_new)
            * [`.updateContentType()`](#module_utils/hub..FileResponse+updateContentType) ⇒ void
            * [`.clone()`](#module_utils/hub..FileResponse+clone) ⇒ FileResponse
            * [`.arrayBuffer()`](#module_utils/hub..FileResponse+arrayBuffer) ⇒ Promise.<ArrayBuffer>
            * [`.blob()`](#module_utils/hub..FileResponse+blob) ⇒ Promise.<Blob>
            * [`.text()`](#module_utils/hub..FileResponse+text) ⇒ Promise.<string>
            * [`.json()`](#module_utils/hub..FileResponse+json) ⇒ Promise.<Object>
        * [~FileCache](#module_utils/hub..FileCache)
            * [`new FileCache(path)`](#new_module_utils/hub..FileCache_new)
            * [`.match(request)`](#module_utils/hub..FileCache+match) ⇒ Promise.<(FileResponse|undefined)>
            * [`.put(request, response, [progress_callback])`](#module_utils/hub..FileCache+put) ⇒ Promise.<void>
        * [`~CONTENT_TYPE_MAP`](#module_utils/hub..CONTENT_TYPE_MAP)
        * [`~isValidUrl(string, [protocols], [validHosts])`](#module_utils/hub..isValidUrl) ⇒ boolean
        * [`~isValidHfModelId(string)`](#module_utils/hub..isValidHfModelId) ⇒ boolean
        * [`~handleError(status, remoteURL, fatal)`](#module_utils/hub..handleError) ⇒ null
        * [`~tryCache(cache, ...names)`](#module_utils/hub..tryCache) ⇒ Promise.<(FileResponse|Response|undefined)>
        * [`~readResponse(response, progress_callback)`](#module_utils/hub..readResponse) ⇒ Promise.<Uint8Array>
        * [`~pathJoin(...parts)`](#module_utils/hub..pathJoin) ⇒ string
        * [`~ExternalData`](#module_utils/hub..ExternalData) : boolean | number
        * [`~PretrainedOptions`](#module_utils/hub..PretrainedOptions) : Object
        * [`~ModelSpecificPretrainedOptions`](#module_utils/hub..ModelSpecificPretrainedOptions) : Object
        * [`~PretrainedModelOptions`](#module_utils/hub..PretrainedModelOptions) : *

* * *

## `utils/hub.getFile(urlOrPath)` ⇒ Promise.<(FileResponse|Response)>

Helper function to get a file, using either the Fetch API or FileSystem API.

**Kind**: static method of [utils/hub](#module_utils/hub)  
**Returns**: Promise.<(FileResponse|Response)> - A promise that resolves to a FileResponse object (if the file is retrieved using the FileSystem API), or a Response object (if the file is retrieved using the Fetch API).  

  
    
      ParamTypeDescription
    
  
  

    urlOrPathURL | stringThe URL/path of the file to get.

      

* * *

## `utils/hub.getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path])` ⇒ Promise.<(string|Uint8Array)>

Retrieves a file from either a remote URL using the Fetch API or from the local file system using the FileSystem API.
If the filesystem is available and `env.useCache = true`, the file will be downloaded and cached.

**Kind**: static method of [utils/hub](#module_utils/hub)  
**Returns**: Promise.<(string|Uint8Array)> - A Promise that resolves with the file content as a Uint8Array if `return_path` is false, or the file path as a string if `return_path` is true.  
**Throws**:

- Will throw an error if the file is not found and `fatal` is true.

  
    
      ParamTypeDefaultDescription
    
  
  

    path_or_repo_idstringThis can be either:

a string, the model id of a model repo on huggingface.co.
a path to a directory potentially containing the file.

    
    filenamestringThe name of the file to locate in path_or_repo.

    
    [fatal]booleantrueWhether to throw an error if the file is not found.

    
    [options]PretrainedOptionsAn object containing optional parameters.

    
    [return_path]booleanfalseWhether to return the path of the file instead of the file content.

      

* [`.getModelFile(path_or_repo_id, filename, [fatal], [options], [return_path])`](#module_utils/hub.getModelFile) ⇒ Promise.<(string|Uint8Array)>
    * [`~cacheKey`](#module_utils/hub.getModelFile..cacheKey) : string
    * [`~response`](#module_utils/hub.getModelFile..response) : Response | FileResponse | undefined
    * [`~buffer`](#module_utils/hub.getModelFile..buffer) : Uint8Array

* * *

### `getModelFile~cacheKey` : string

**Kind**: inner property of [getModelFile](#module_utils/hub.getModelFile)  

* * *

### `getModelFile~response` : Response | FileResponse | undefined

**Kind**: inner property of [getModelFile](#module_utils/hub.getModelFile)  

* * *

### `getModelFile~buffer` : Uint8Array

**Kind**: inner property of [getModelFile](#module_utils/hub.getModelFile)  

* * *

## `utils/hub.getModelText(modelPath, fileName, [fatal], [options])` ⇒ Promise.<(string|null)>

Fetches a text file from a given path and file name.

**Kind**: static method of [utils/hub](#module_utils/hub)  
**Returns**: Promise.<(string|null)> - The text content of the file.  
**Throws**:

- Will throw an error if the file is not found and `fatal` is true.

  
    
      ParamTypeDefaultDescription
    
  
  

    modelPathstringThe path to the directory containing the file.

    
    fileNamestringThe name of the file to fetch.

    
    [fatal]booleantrueWhether to throw an error if the file is not found.

    
    [options]PretrainedOptionsAn object containing optional parameters.

      

* * *

## `utils/hub.getModelJSON(modelPath, fileName, [fatal], [options])` ⇒ Promise.<Object>

Fetches a JSON file from a given path and file name.

**Kind**: static method of [utils/hub](#module_utils/hub)  
**Returns**: Promise.<Object> - The JSON data parsed into a JavaScript object.  
**Throws**:

- Will throw an error if the file is not found and `fatal` is true.

  
    
      ParamTypeDefaultDescription
    
  
  

    modelPathstringThe path to the directory containing the file.

    
    fileNamestringThe name of the file to fetch.

    
    [fatal]booleantrueWhether to throw an error if the file is not found.

    
    [options]PretrainedOptionsAn object containing optional parameters.

      

* * *

## utils/hub~FileResponse

**Kind**: inner class of [utils/hub](#module_utils/hub)  

* [~FileResponse](#module_utils/hub..FileResponse)
    * [`new FileResponse(filePath)`](#new_module_utils/hub..FileResponse_new)
    * [`.updateContentType()`](#module_utils/hub..FileResponse+updateContentType) ⇒ void
    * [`.clone()`](#module_utils/hub..FileResponse+clone) ⇒ FileResponse
    * [`.arrayBuffer()`](#module_utils/hub..FileResponse+arrayBuffer) ⇒ Promise.<ArrayBuffer>
    * [`.blob()`](#module_utils/hub..FileResponse+blob) ⇒ Promise.<Blob>
    * [`.text()`](#module_utils/hub..FileResponse+text) ⇒ Promise.<string>
    * [`.json()`](#module_utils/hub..FileResponse+json) ⇒ Promise.<Object>

* * *

### `new FileResponse(filePath)`

Creates a new `FileResponse` object.

  
    
      ParamType
    
  
  

    filePathstring
      

* * *

### `fileResponse.updateContentType()` ⇒ void

Updates the 'content-type' header property of the response based on the extension of
the file specified by the filePath property of the current object.

**Kind**: instance method of [FileResponse](#module_utils/hub..FileResponse)  

* * *

### `fileResponse.clone()` ⇒ FileResponse

Clone the current FileResponse object.

**Kind**: instance method of [FileResponse](#module_utils/hub..FileResponse)  
**Returns**: FileResponse - A new FileResponse object with the same properties as the current object.  

* * *

### `fileResponse.arrayBuffer()` ⇒ Promise.<ArrayBuffer>

Reads the contents of the file specified by the filePath property and returns a Promise that
resolves with an ArrayBuffer containing the file's contents.

**Kind**: instance method of [FileResponse](#module_utils/hub..FileResponse)  
**Returns**: Promise.<ArrayBuffer> - A Promise that resolves with an ArrayBuffer containing the file's contents.  
**Throws**:

- Error If the file cannot be read.

* * *

### `fileResponse.blob()` ⇒ Promise.<Blob>

Reads the contents of the file specified by the filePath property and returns a Promise that
resolves with a Blob containing the file's contents.

**Kind**: instance method of [FileResponse](#module_utils/hub..FileResponse)  
**Returns**: Promise.<Blob> - A Promise that resolves with a Blob containing the file's contents.  
**Throws**:

- Error If the file cannot be read.

* * *

### `fileResponse.text()` ⇒ Promise.<string>

Reads the contents of the file specified by the filePath property and returns a Promise that
resolves with a string containing the file's contents.

**Kind**: instance method of [FileResponse](#module_utils/hub..FileResponse)  
**Returns**: Promise.<string> - A Promise that resolves with a string containing the file's contents.  
**Throws**:

- Error If the file cannot be read.

* * *

### `fileResponse.json()` ⇒ Promise.<Object>

Reads the contents of the file specified by the filePath property and returns a Promise that
resolves with a parsed JavaScript object containing the file's contents.

**Kind**: instance method of [FileResponse](#module_utils/hub..FileResponse)  
**Returns**: Promise.<Object> - A Promise that resolves with a parsed JavaScript object containing the file's contents.  
**Throws**:

- Error If the file cannot be read.

* * *

## utils/hub~FileCache

**Kind**: inner class of [utils/hub](#module_utils/hub)  

* [~FileCache](#module_utils/hub..FileCache)
    * [`new FileCache(path)`](#new_module_utils/hub..FileCache_new)
    * [`.match(request)`](#module_utils/hub..FileCache+match) ⇒ Promise.<(FileResponse|undefined)>
    * [`.put(request, response, [progress_callback])`](#module_utils/hub..FileCache+put) ⇒ Promise.<void>

* * *

### `new FileCache(path)`

Instantiate a `FileCache` object.

  
    
      ParamType
    
  
  

    pathstring
      

* * *

### `fileCache.match(request)` ⇒ Promise.<(FileResponse|undefined)>

Checks whether the given request is in the cache.

**Kind**: instance method of [FileCache](#module_utils/hub..FileCache)  

  
    
      ParamType
    
  
  

    requeststring
      

* * *

### `fileCache.put(request, response, [progress_callback])` ⇒ Promise.<void>

Adds the given response to the cache.

**Kind**: instance method of [FileCache](#module_utils/hub..FileCache)  

  
    
      ParamTypeDescription
    
  
  

    requeststring
    
    responseResponse
    
    [progress_callback]*Optional.
The function to call with progress updates

      

* * *

## `utils/hub~CONTENT_TYPE_MAP`

Mapping from file extensions to MIME types.

**Kind**: inner constant of [utils/hub](#module_utils/hub)  

* * *

## `utils/hub~isValidUrl(string, [protocols], [validHosts])` ⇒ boolean

Determines whether the given string is a valid URL.

**Kind**: inner method of [utils/hub](#module_utils/hub)  
**Returns**: boolean - True if the string is a valid URL, false otherwise.  

  
    
      ParamTypeDefaultDescription
    
  
  

    stringstring | URLThe string to test for validity as an URL.

    
    [protocols]Array.<string>A list of valid protocols. If specified, the protocol must be in this list.

    
    [validHosts]Array.<string>A list of valid hostnames. If specified, the URL's hostname must be in this list.

      

* * *

## `utils/hub~isValidHfModelId(string)` ⇒ boolean

Tests whether a string is a valid Hugging Face model ID or not.
Adapted from https://github.com/huggingface/huggingface_hub/blob/6378820ebb03f071988a96c7f3268f5bdf8f9449/src/huggingface_hub/utils/_validators.py#L119-L170

**Kind**: inner method of [utils/hub](#module_utils/hub)  
**Returns**: boolean - True if the string is a valid model ID, false otherwise.  

  
    
      ParamTypeDescription
    
  
  

    stringstringThe string to test

      

* * *

## `utils/hub~handleError(status, remoteURL, fatal)` ⇒ null

Helper method to handle fatal errors that occur while trying to load a file from the Hugging Face Hub.

**Kind**: inner method of [utils/hub](#module_utils/hub)  
**Returns**: null - Returns `null` if `fatal = true`.  
**Throws**:

- Error If `fatal = false`.

  
    
      ParamTypeDescription
    
  
  

    statusnumberThe HTTP status code of the error.

    
    remoteURLstringThe URL of the file that could not be loaded.

    
    fatalbooleanWhether to raise an error if the file could not be loaded.

      

* * *

## `utils/hub~tryCache(cache, ...names)` ⇒ Promise.<(FileResponse|Response|undefined)>

**Kind**: inner method of [utils/hub](#module_utils/hub)  
**Returns**: Promise.<(FileResponse|Response|undefined)> - The item from the cache, or undefined if not found.  

  
    
      ParamTypeDescription
    
  
  

    cacheFileCache | CacheThe cache to search

    
    ...namesArray.<string>The names of the item to search for

      

* * *

## `utils/hub~readResponse(response, progress_callback)` ⇒ Promise.<Uint8Array>

Read and track progress when reading a Response object

**Kind**: inner method of [utils/hub](#module_utils/hub)  
**Returns**: Promise.<Uint8Array> - A Promise that resolves with the Uint8Array buffer  

  
    
      ParamTypeDescription
    
  
  

    responseResponse | FileResponseThe Response object to read

    
    progress_callback*The function to call with progress updates

      

* * *

## `utils/hub~pathJoin(...parts)` ⇒ string

Joins multiple parts of a path into a single path, while handling leading and trailing slashes.

**Kind**: inner method of [utils/hub](#module_utils/hub)  
**Returns**: string - A string representing the joined path.  

  
    
      ParamTypeDescription
    
  
  

    ...partsstringMultiple parts of a path.

      

* * *

## `utils/hub~ExternalData` : boolean | number

Whether to load the model using the external data format (used for models >= 2GB in size).
If `true`, the model will be loaded using the external data format.
If a number, this many chunks will be loaded using the external data format (of the form: "model.onnx_data[_{chunk_number}]").

**Kind**: inner typedef of [utils/hub](#module_utils/hub)  

* * *

## `utils/hub~PretrainedOptions` : Object

Options for loading a pretrained model.

**Kind**: inner typedef of [utils/hub](#module_utils/hub)  
**Properties**

  
    
      NameTypeDefaultDescription
    
  
  

    [progress_callback]*If specified, this function will be called during model construction, to provide the user with progress updates.

    
    [config]*Configuration for the model to use instead of an automatically loaded configuration. Configuration can be automatically loaded when:

The model is a model provided by the library (loaded with the model id string of a pretrained model).
The model is loaded by supplying a local directory as pretrained_model_name_or_path and a configuration JSON file named config.json is found in the directory.

    
    [cache_dir]stringnullPath to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.

    
    [local_files_only]booleanfalseWhether or not to only look at local files (e.g., not try downloading the model).

    
    [revision]string"'main'"The specific model version to use. It can be a branch name, a tag name, or a commit id,
since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.
NOTE: This setting is ignored for local requests.

      

* * *

## `utils/hub~ModelSpecificPretrainedOptions` : Object

Options for loading a pretrained model.

**Kind**: inner typedef of [utils/hub](#module_utils/hub)  
**Properties**

  
    
      NameTypeDefaultDescription
    
  
  

    [subfolder]string"'onnx'"In case the relevant files are located inside a subfolder of the model repo on huggingface.co,
you can specify the folder name here.

    
    [model_file_name]stringnullIf specified, load the model with this name (excluding the .onnx suffix). Currently only valid for encoder- or decoder-only models.

    
    [device]*The device to run the model on. If not specified, the device will be chosen from the environment settings.

    
    [dtype]*The data type to use for the model. If not specified, the data type will be chosen from the environment settings.

    
    [use_external_data_format]ExternalData | Record<string, ExternalData>falseWhether to load the model using the external data format (used for models >= 2GB in size).

    
    [session_options]*(Optional) User-specified session options passed to the runtime. If not provided, suitable defaults will be chosen.

      

* * *

## `utils/hub~PretrainedModelOptions` : *

Options for loading a pretrained model.

**Kind**: inner typedef of [utils/hub](#module_utils/hub)  

* * *