fhnw
/

fhnw-tiktoken-tokenizer

Model card Files Files and versions

fhnw-tiktoken-tokenizer / tokenizer.pkl

andreasmartin's picture

Upload tokenizer.pkl with huggingface_hub

87abe53 verified 6 months ago

history blame contribute delete

2.19 MB

This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )

Xet hash:: 980f775b39a9c1c484401bb5ec8f711a12124efbab7cf1e18aa443d79504c73e
Size of remote file:: 2.19 MB
SHA256:: 49063b3f76aca6f78a9ede1423a4bb86dd06a505d0025b70a65aea9cc9bcc7b6

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.