Buckets:

hf-doc-build
/

doc

Files

xet

hf-doc-build/doc / transformers.js /main /en /api /utils /core.md

rtrm

about 1 month ago

preview code

download

raw

11.2 kB

utils/core

Core utility functions/classes for Transformers.js.

These are only used internally, meaning an end-user shouldn't need to access anything here.

utils/core
- static
  - .DefaultProgressCallback
    - new DefaultProgressCallback(callback, files_loading)
    - ._call(info)
  - .reverseDictionary(data) ⇒ Object
  - .escapeRegExp(string) ⇒ string
  - .isTypedArray(val) ⇒ boolean
  - .isIntegralNumber(x) ⇒ boolean
  - .isNullishDimension(x) ⇒ boolean
  - .calculateDimensions(arr) ⇒ Array
  - .pop(obj, key, defaultValue) ⇒ *
  - .mergeArrays(arrs) ⇒ Array
  - .calculateReflectOffset(i, w) ⇒ number
  - .pick(o, props) ⇒ Object
  - .len(s) ⇒ number
  - .count(arr, value)
- inner
  - ~InitiateProgressInfo : Object
  - ~DownloadProgressInfo : Object
  - ~ProgressStatusInfo : Object
  - ~FileLoadingProgress : Object
  - ~FilesLoadingMap : Record.<string, FileLoadingProgress>
  - ~TotalProgressInfo : Object
  - ~DoneProgressInfo : Object
  - ~ReadyProgressInfo : Object
  - ~ProgressInfo : InitiateProgressInfo | DownloadProgressInfo | ProgressStatusInfo | DoneProgressInfo | ReadyProgressInfo | TotalProgressInfo
  - ~ProgressCallback ⇒ void

utils/core.DefaultProgressCallback

A callable progress callback that wraps an original callback and emits aggregate progress_total events. Because it extends Callable, instances can be passed directly wherever a plain callback function is expected.

Callers can check callback instanceof DefaultProgressCallback to avoid double-wrapping when both pipeline() and from_pretrained() would otherwise each add their own wrapper.

Kind: static class of utils/core

.DefaultProgressCallback
- new DefaultProgressCallback(callback, files_loading)
- ._call(info)

`new DefaultProgressCallback(callback, files_loading)`

  ParamTypeDescription




callbackProgressCallbackThe original callback.


files_loadingFilesLoadingMapMutable map storing per-file progress.

`defaultProgressCallback._call(info)`

Kind: instance method of DefaultProgressCallback

  ParamType




infoProgressInfo

`utils/core.reverseDictionary(data)` ⇒ Object

Reverses the keys and values of an object.

Kind: static method of utils/core
Returns: Object - The reversed object.
See: https://ultimatecourses.com/blog/reverse-object-keys-and-values-in-javascript

  ParamTypeDescription




dataObjectThe object to reverse.

`utils/core.escapeRegExp(string)` ⇒ string

Escapes regular expression special characters from a string by replacing them with their escaped counterparts.

Kind: static method of utils/core
Returns: string - The escaped string.

  ParamTypeDescription




stringstringThe string to escape.

`utils/core.isTypedArray(val)` ⇒ boolean

Check if a value is a typed array.

Kind: static method of utils/core
Returns: boolean - True if the value is a TypedArray, false otherwise.

Adapted from https://stackoverflow.com/a/71091338/13989043

  ParamTypeDescription




val*The value to check.

`utils/core.isIntegralNumber(x)` ⇒ boolean

Check if a value is an integer.

Kind: static method of utils/core
Returns: boolean - True if the value is a string, false otherwise.

  ParamTypeDescription




x*The value to check.

`utils/core.isNullishDimension(x)` ⇒ boolean

Determine if a provided width or height is nullish.

Kind: static method of utils/core
Returns: boolean - True if the value is null, undefined or -1, false otherwise.

  ParamTypeDescription




x*The value to check.

`utils/core.calculateDimensions(arr)` ⇒ Array

Calculates the dimensions of a nested array.

Kind: static method of utils/core
Returns: Array - An array containing the dimensions of the input array.

  ParamTypeDescription




arrArrayThe nested array to calculate dimensions for.

`utils/core.pop(obj, key, defaultValue)` ⇒ *

Replicate python's .pop() method for objects.

Kind: static method of utils/core
Returns: * - The value of the popped key.
Throws:

Error If the key does not exist and no default value is provided.
```
ParamTypeDescription
```
objObjectThe object to pop from.

keystringThe key to pop.

defaultValue*The default value to return if the key does not exist.

`utils/core.mergeArrays(arrs)` ⇒ Array

Efficiently merge arrays, creating a new copy. Adapted from https://stackoverflow.com/a/6768642/13989043

Kind: static method of utils/core
Returns: Array - The merged array.

  ParamTypeDescription




arrsArrayArrays to merge.

`utils/core.calculateReflectOffset(i, w)` ⇒ number

Calculates the index offset for a given index and window size.

Kind: static method of utils/core
Returns: number - The index offset.

  ParamTypeDescription




inumberThe index.


wnumberThe window size.

`utils/core.pick(o, props)` ⇒ Object

Kind: static method of utils/core

  ParamType




oObject

propsArray

`utils/core.len(s)` ⇒ number

Calculate the length of a string, taking multi-byte characters into account. This mimics the behavior of Python's len function.

Kind: static method of utils/core
Returns: number - The length of the string.

  ParamTypeDescription




sstringThe string to calculate the length of.

`utils/core.count(arr, value)`

Count the occurrences of a value in an array or string. This mimics the behavior of Python's count method.

Kind: static method of utils/core

  ParamTypeDescription




arrArray | stringThe array or string to search.


valueanyThe value to count.