transformers.js/pr_1452/en/api/utils/data-structures.md

# utils/data-structures

Custom data structures.

These are only used internally, meaning an end-user shouldn't
need to access anything here.


* [utils/data-structures](#module_utils/data-structures)
    * _static_
        * [.PriorityQueue](#module_utils/data-structures.PriorityQueue)
            * [`new PriorityQueue(comparator)`](#new_module_utils/data-structures.PriorityQueue_new)
            * [`.size`](#module_utils/data-structures.PriorityQueue+size)
            * [`.isEmpty()`](#module_utils/data-structures.PriorityQueue+isEmpty) ⇒ <code>boolean</code>
            * [`.peek()`](#module_utils/data-structures.PriorityQueue+peek) ⇒ <code>any</code>
            * [`.push(...values)`](#module_utils/data-structures.PriorityQueue+push) ⇒ <code>number</code>
            * [`.extend(values)`](#module_utils/data-structures.PriorityQueue+extend) ⇒ <code>number</code>
            * [`.pop()`](#module_utils/data-structures.PriorityQueue+pop) ⇒ <code>any</code>
            * [`.replace(value)`](#module_utils/data-structures.PriorityQueue+replace) ⇒ <code>*</code>
            * [`._siftUpFrom(node)`](#module_utils/data-structures.PriorityQueue+_siftUpFrom)
        * [.CharTrie](#module_utils/data-structures.CharTrie)
            * [`.extend(texts)`](#module_utils/data-structures.CharTrie+extend)
            * [`.push(text)`](#module_utils/data-structures.CharTrie+push)
            * [`.commonPrefixSearch(text)`](#module_utils/data-structures.CharTrie+commonPrefixSearch)
        * [.TokenLattice](#module_utils/data-structures.TokenLattice)
            * [`new TokenLattice(sentence, bosTokenId, eosTokenId)`](#new_module_utils/data-structures.TokenLattice_new)
            * [`.insert(pos, length, score, tokenId)`](#module_utils/data-structures.TokenLattice+insert)
            * [`.viterbi()`](#module_utils/data-structures.TokenLattice+viterbi) ⇒ <code>Array.&lt;TokenLatticeNode&gt;</code>
            * [`.piece(node)`](#module_utils/data-structures.TokenLattice+piece) ⇒ <code>string</code>
            * [`.tokens()`](#module_utils/data-structures.TokenLattice+tokens) ⇒ <code>Array.&lt;string&gt;</code>
            * [`.tokenIds()`](#module_utils/data-structures.TokenLattice+tokenIds) ⇒ <code>Array.&lt;number&gt;</code>
        * [.DictionarySplitter](#module_utils/data-structures.DictionarySplitter)
            * [`new DictionarySplitter(dictionary)`](#new_module_utils/data-structures.DictionarySplitter_new)
            * [`.split(text)`](#module_utils/data-structures.DictionarySplitter+split) ⇒ <code>Array.&lt;string&gt;</code>
        * [.LRUCache](#module_utils/data-structures.LRUCache)
            * [`new LRUCache(capacity)`](#new_module_utils/data-structures.LRUCache_new)
            * [`.get(key)`](#module_utils/data-structures.LRUCache+get) ⇒ <code>any</code>
            * [`.put(key, value)`](#module_utils/data-structures.LRUCache+put)
            * [`.clear()`](#module_utils/data-structures.LRUCache+clear)
    * _inner_
        * [~CharTrieNode](#module_utils/data-structures..CharTrieNode)
            * [`new CharTrieNode(isLeaf, children)`](#new_module_utils/data-structures..CharTrieNode_new)
            * [`.default()`](#module_utils/data-structures..CharTrieNode.default) ⇒ <code>CharTrieNode</code>
        * [~TokenLatticeNode](#module_utils/data-structures..TokenLatticeNode)
            * [`new TokenLatticeNode(tokenId, nodeId, pos, length, score)`](#new_module_utils/data-structures..TokenLatticeNode_new)
            * [`.clone()`](#module_utils/data-structures..TokenLatticeNode+clone) ⇒ <code>TokenLatticeNode</code>


* * *

<a id="module_utils/data-structures.PriorityQueue" class="group"></a>

## utils/data-structures.PriorityQueue

Efficient Heap-based Implementation of a Priority Queue.
It uses an array-based binary heap, where the root is at index `0`, and the
children of node `i` are located at indices `2i + 1` and `2i + 2`, respectively.

Adapted from the following sources:
- https://stackoverflow.com/a/42919752/13989043 (original)
- https://github.com/belladoreai/llama-tokenizer-js (minor improvements)

**Kind**: static class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [.PriorityQueue](#module_utils/data-structures.PriorityQueue)
    * [`new PriorityQueue(comparator)`](#new_module_utils/data-structures.PriorityQueue_new)
    * [`.size`](#module_utils/data-structures.PriorityQueue+size)
    * [`.isEmpty()`](#module_utils/data-structures.PriorityQueue+isEmpty) ⇒ <code>boolean</code>
    * [`.peek()`](#module_utils/data-structures.PriorityQueue+peek) ⇒ <code>any</code>
    * [`.push(...values)`](#module_utils/data-structures.PriorityQueue+push) ⇒ <code>number</code>
    * [`.extend(values)`](#module_utils/data-structures.PriorityQueue+extend) ⇒ <code>number</code>
    * [`.pop()`](#module_utils/data-structures.PriorityQueue+pop) ⇒ <code>any</code>
    * [`.replace(value)`](#module_utils/data-structures.PriorityQueue+replace) ⇒ <code>*</code>
    * [`._siftUpFrom(node)`](#module_utils/data-structures.PriorityQueue+_siftUpFrom)


* * *

<a id="new_module_utils/data-structures.PriorityQueue_new" class="group"></a>

### `new PriorityQueue(comparator)`

Create a new PriorityQueue.

<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>comparator</td><td><code>function</code></td><td><p>Comparator function to determine priority. Defaults to a MaxHeap.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.PriorityQueue+size" class="group"></a>

### `priorityQueue.size`

The size of the queue

**Kind**: instance property of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  

* * *

<a id="module_utils/data-structures.PriorityQueue+isEmpty" class="group"></a>

### `priorityQueue.isEmpty()` ⇒ <code>boolean</code>

Check if the queue is empty.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
**Returns**: <code>boolean</code> - `true` if the queue is empty, `false` otherwise.  

* * *

<a id="module_utils/data-structures.PriorityQueue+peek" class="group"></a>

### `priorityQueue.peek()` ⇒ <code>any</code>

Return the element with the highest priority in the queue.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
**Returns**: <code>any</code> - The highest priority element in the queue.  

* * *

<a id="module_utils/data-structures.PriorityQueue+push" class="group"></a>

### `priorityQueue.push(...values)` ⇒ <code>number</code>

Add one or more elements to the queue.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
**Returns**: <code>number</code> - The new size of the queue.  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>...values</td><td><code>any</code></td><td><p>The values to push into the queue.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.PriorityQueue+extend" class="group"></a>

### `priorityQueue.extend(values)` ⇒ <code>number</code>

Add multiple elements to the queue.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
**Returns**: <code>number</code> - The new size of the queue.  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>values</td><td><code>Array.&lt;any&gt;</code></td><td><p>The values to push into the queue.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.PriorityQueue+pop" class="group"></a>

### `priorityQueue.pop()` ⇒ <code>any</code>

Remove and return the element with the highest priority in the queue.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
**Returns**: <code>any</code> - The element with the highest priority in the queue.  

* * *

<a id="module_utils/data-structures.PriorityQueue+replace" class="group"></a>

### `priorityQueue.replace(value)` ⇒ <code>*</code>

Replace the element with the highest priority in the queue with a new value.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
**Returns**: <code>*</code> - The replaced value.  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>value</td><td><code>*</code></td><td><p>The new value.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.PriorityQueue+_siftUpFrom" class="group"></a>

### `priorityQueue._siftUpFrom(node)`

Helper function to sift up from a given node.

**Kind**: instance method of [<code>PriorityQueue</code>](#module_utils/data-structures.PriorityQueue)  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>node</td><td><code>number</code></td><td><p>The index of the node to start sifting up from.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.CharTrie" class="group"></a>

## utils/data-structures.CharTrie

A trie structure to efficiently store and search for strings.

**Kind**: static class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [.CharTrie](#module_utils/data-structures.CharTrie)
    * [`.extend(texts)`](#module_utils/data-structures.CharTrie+extend)
    * [`.push(text)`](#module_utils/data-structures.CharTrie+push)
    * [`.commonPrefixSearch(text)`](#module_utils/data-structures.CharTrie+commonPrefixSearch)


* * *

<a id="module_utils/data-structures.CharTrie+extend" class="group"></a>

### `charTrie.extend(texts)`

Adds one or more `texts` to the trie.

**Kind**: instance method of [<code>CharTrie</code>](#module_utils/data-structures.CharTrie)  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>texts</td><td><code>Array.&lt;string&gt;</code></td><td><p>The strings to add to the trie.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.CharTrie+push" class="group"></a>

### `charTrie.push(text)`

Adds text to the trie.

**Kind**: instance method of [<code>CharTrie</code>](#module_utils/data-structures.CharTrie)  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>text</td><td><code>string</code></td><td><p>The string to add to the trie.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.CharTrie+commonPrefixSearch" class="group"></a>

### `charTrie.commonPrefixSearch(text)`

Searches the trie for all strings with a common prefix of `text`.

**Kind**: instance method of [<code>CharTrie</code>](#module_utils/data-structures.CharTrie)  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>text</td><td><code>string</code></td><td><p>The common prefix to search for.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.TokenLattice" class="group"></a>

## utils/data-structures.TokenLattice

A lattice data structure to be used for tokenization.

**Kind**: static class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [.TokenLattice](#module_utils/data-structures.TokenLattice)
    * [`new TokenLattice(sentence, bosTokenId, eosTokenId)`](#new_module_utils/data-structures.TokenLattice_new)
    * [`.insert(pos, length, score, tokenId)`](#module_utils/data-structures.TokenLattice+insert)
    * [`.viterbi()`](#module_utils/data-structures.TokenLattice+viterbi) ⇒ <code>Array.&lt;TokenLatticeNode&gt;</code>
    * [`.piece(node)`](#module_utils/data-structures.TokenLattice+piece) ⇒ <code>string</code>
    * [`.tokens()`](#module_utils/data-structures.TokenLattice+tokens) ⇒ <code>Array.&lt;string&gt;</code>
    * [`.tokenIds()`](#module_utils/data-structures.TokenLattice+tokenIds) ⇒ <code>Array.&lt;number&gt;</code>


* * *

<a id="new_module_utils/data-structures.TokenLattice_new" class="group"></a>

### `new TokenLattice(sentence, bosTokenId, eosTokenId)`

Creates a new TokenLattice instance.

<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>sentence</td><td><code>string</code></td><td><p>The input sentence to be tokenized.</p>
</td>
    </tr><tr>
    <td>bosTokenId</td><td><code>number</code></td><td><p>The beginning-of-sequence token ID.</p>
</td>
    </tr><tr>
    <td>eosTokenId</td><td><code>number</code></td><td><p>The end-of-sequence token ID.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.TokenLattice+insert" class="group"></a>

### `tokenLattice.insert(pos, length, score, tokenId)`

Inserts a new token node into the token lattice.

**Kind**: instance method of [<code>TokenLattice</code>](#module_utils/data-structures.TokenLattice)  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>pos</td><td><code>number</code></td><td><p>The starting position of the token.</p>
</td>
    </tr><tr>
    <td>length</td><td><code>number</code></td><td><p>The length of the token.</p>
</td>
    </tr><tr>
    <td>score</td><td><code>number</code></td><td><p>The score of the token.</p>
</td>
    </tr><tr>
    <td>tokenId</td><td><code>number</code></td><td><p>The token ID of the token.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.TokenLattice+viterbi" class="group"></a>

### `tokenLattice.viterbi()` ⇒ <code>Array.&lt;TokenLatticeNode&gt;</code>

Implements the Viterbi algorithm to compute the most likely sequence of tokens.

**Kind**: instance method of [<code>TokenLattice</code>](#module_utils/data-structures.TokenLattice)  
**Returns**: <code>Array.&lt;TokenLatticeNode&gt;</code> - The most likely sequence of tokens.  

* * *

<a id="module_utils/data-structures.TokenLattice+piece" class="group"></a>

### `tokenLattice.piece(node)` ⇒ <code>string</code>

**Kind**: instance method of [<code>TokenLattice</code>](#module_utils/data-structures.TokenLattice)  
**Returns**: <code>string</code> - The array of nodes representing the most likely sequence of tokens.  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>node</td><td><code>TokenLatticeNode</code></td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.TokenLattice+tokens" class="group"></a>

### `tokenLattice.tokens()` ⇒ <code>Array.&lt;string&gt;</code>

**Kind**: instance method of [<code>TokenLattice</code>](#module_utils/data-structures.TokenLattice)  
**Returns**: <code>Array.&lt;string&gt;</code> - The most likely sequence of tokens.  

* * *

<a id="module_utils/data-structures.TokenLattice+tokenIds" class="group"></a>

### `tokenLattice.tokenIds()` ⇒ <code>Array.&lt;number&gt;</code>

**Kind**: instance method of [<code>TokenLattice</code>](#module_utils/data-structures.TokenLattice)  
**Returns**: <code>Array.&lt;number&gt;</code> - The most likely sequence of token ids.  

* * *

<a id="module_utils/data-structures.DictionarySplitter" class="group"></a>

## utils/data-structures.DictionarySplitter

A data structure which uses a trie to split a string into tokens based on a dictionary.
It can also use a regular expression to preprocess the input text before splitting.

NOTE: To ensure multi-byte characters are handled correctly, we operate at byte-level instead of character-level.

**Kind**: static class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [.DictionarySplitter](#module_utils/data-structures.DictionarySplitter)
    * [`new DictionarySplitter(dictionary)`](#new_module_utils/data-structures.DictionarySplitter_new)
    * [`.split(text)`](#module_utils/data-structures.DictionarySplitter+split) ⇒ <code>Array.&lt;string&gt;</code>


* * *

<a id="new_module_utils/data-structures.DictionarySplitter_new" class="group"></a>

### `new DictionarySplitter(dictionary)`

<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>dictionary</td><td><code>Array.&lt;string&gt;</code></td><td><p>The dictionary of words to use for splitting.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.DictionarySplitter+split" class="group"></a>

### `dictionarySplitter.split(text)` ⇒ <code>Array.&lt;string&gt;</code>

Splits the input text into tokens based on the dictionary.

**Kind**: instance method of [<code>DictionarySplitter</code>](#module_utils/data-structures.DictionarySplitter)  
**Returns**: <code>Array.&lt;string&gt;</code> - An array of tokens.  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>text</td><td><code>string</code></td><td><p>The input text to split.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.LRUCache" class="group"></a>

## utils/data-structures.LRUCache

A simple Least Recently Used (LRU) cache implementation in JavaScript.
This cache stores key-value pairs and evicts the least recently used item
when the capacity is exceeded.

**Kind**: static class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [.LRUCache](#module_utils/data-structures.LRUCache)
    * [`new LRUCache(capacity)`](#new_module_utils/data-structures.LRUCache_new)
    * [`.get(key)`](#module_utils/data-structures.LRUCache+get) ⇒ <code>any</code>
    * [`.put(key, value)`](#module_utils/data-structures.LRUCache+put)
    * [`.clear()`](#module_utils/data-structures.LRUCache+clear)


* * *

<a id="new_module_utils/data-structures.LRUCache_new" class="group"></a>

### `new LRUCache(capacity)`

Creates an LRUCache instance.

<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>capacity</td><td><code>number</code></td><td><p>The maximum number of items the cache can hold.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.LRUCache+get" class="group"></a>

### `lruCache.get(key)` ⇒ <code>any</code>

Retrieves the value associated with the given key and marks the key as recently used.

**Kind**: instance method of [<code>LRUCache</code>](#module_utils/data-structures.LRUCache)  
**Returns**: <code>any</code> - The value associated with the key, or undefined if the key does not exist.  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>key</td><td><code>any</code></td><td><p>The key to retrieve.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.LRUCache+put" class="group"></a>

### `lruCache.put(key, value)`

Inserts or updates the key-value pair in the cache.
If the key already exists, it is updated and marked as recently used.
If the cache exceeds its capacity, the least recently used item is evicted.

**Kind**: instance method of [<code>LRUCache</code>](#module_utils/data-structures.LRUCache)  
<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>key</td><td><code>any</code></td><td><p>The key to add or update.</p>
</td>
    </tr><tr>
    <td>value</td><td><code>any</code></td><td><p>The value to associate with the key.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures.LRUCache+clear" class="group"></a>

### `lruCache.clear()`

Clears the cache.

**Kind**: instance method of [<code>LRUCache</code>](#module_utils/data-structures.LRUCache)  

* * *

<a id="module_utils/data-structures..CharTrieNode" class="group"></a>

## utils/data-structures~CharTrieNode

Represents a node in a character trie.

**Kind**: inner class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [~CharTrieNode](#module_utils/data-structures..CharTrieNode)
    * [`new CharTrieNode(isLeaf, children)`](#new_module_utils/data-structures..CharTrieNode_new)
    * [`.default()`](#module_utils/data-structures..CharTrieNode.default) ⇒ <code>CharTrieNode</code>


* * *

<a id="new_module_utils/data-structures..CharTrieNode_new" class="group"></a>

### `new CharTrieNode(isLeaf, children)`

Create a new CharTrieNode.

<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>isLeaf</td><td><code>boolean</code></td><td><p>Whether the node is a leaf node or not.</p>
</td>
    </tr><tr>
    <td>children</td><td><code>Map.&lt;string, CharTrieNode&gt;</code></td><td><p>A map containing the node&#39;s children, where the key is a character and the value is a <code>CharTrieNode</code>.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures..CharTrieNode.default" class="group"></a>

### `CharTrieNode.default()` ⇒ <code>CharTrieNode</code>

Returns a new `CharTrieNode` instance with default values.

**Kind**: static method of [<code>CharTrieNode</code>](#module_utils/data-structures..CharTrieNode)  
**Returns**: <code>CharTrieNode</code> - A new `CharTrieNode` instance with `isLeaf` set to `false` and an empty `children` map.  

* * *

<a id="module_utils/data-structures..TokenLatticeNode" class="group"></a>

## utils/data-structures~TokenLatticeNode

**Kind**: inner class of [<code>utils/data-structures</code>](#module_utils/data-structures)  

* [~TokenLatticeNode](#module_utils/data-structures..TokenLatticeNode)
    * [`new TokenLatticeNode(tokenId, nodeId, pos, length, score)`](#new_module_utils/data-structures..TokenLatticeNode_new)
    * [`.clone()`](#module_utils/data-structures..TokenLatticeNode+clone) ⇒ <code>TokenLatticeNode</code>


* * *

<a id="new_module_utils/data-structures..TokenLatticeNode_new" class="group"></a>

### `new TokenLatticeNode(tokenId, nodeId, pos, length, score)`

Represents a node in a token lattice for a given sentence.

<table>
  <thead>
    <tr>
      <th>Param</th><th>Type</th><th>Description</th>
    </tr>
  </thead>
  <tbody>
<tr>
    <td>tokenId</td><td><code>number</code></td><td><p>The ID of the token associated with this node.</p>
</td>
    </tr><tr>
    <td>nodeId</td><td><code>number</code></td><td><p>The ID of this node.</p>
</td>
    </tr><tr>
    <td>pos</td><td><code>number</code></td><td><p>The starting position of the token in the sentence.</p>
</td>
    </tr><tr>
    <td>length</td><td><code>number</code></td><td><p>The length of the token.</p>
</td>
    </tr><tr>
    <td>score</td><td><code>number</code></td><td><p>The score associated with the token.</p>
</td>
    </tr>  </tbody>
</table>


* * *

<a id="module_utils/data-structures..TokenLatticeNode+clone" class="group"></a>

### `tokenLatticeNode.clone()` ⇒ <code>TokenLatticeNode</code>

Returns a clone of this node.

**Kind**: instance method of [<code>TokenLatticeNode</code>](#module_utils/data-structures..TokenLatticeNode)  
**Returns**: <code>TokenLatticeNode</code> - A clone of this node.  

* * *



<EditOnGithub source="https://github.com/huggingface/transformers.js/blob/main/docs/source/api/utils/data-structures.md" />