| Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms | |
| --- | |
| MODIFICATION NOTICE | |
| `gemma2.spm` in this repository is a MODIFIED form of Google's Gemma 2 | |
| `tokenizer.model`. It is not the original file. | |
| What was changed: the SentencePiece protocol-buffer model was converted, id by | |
| id, into a plain-text format — one line per token id, holding that id's piece, | |
| score and piece type. Nothing was added, removed, reordered or rounded: all | |
| 256,000 pieces keep their ids, their scores and their SentencePiece piece types, | |
| and the conversion is checked in both directions before the file is written. | |
| What was NOT changed: no vocabulary entry, score, piece type or id assignment | |
| differs from Google's file in any way. | |
| Source: Google's Gemma 2 `tokenizer.model`, MD5 f9e2445870ec741aa6346bbd75531bb4 | |
| --- | |
| Use of Gemma and of Model Derivatives is subject to the use restrictions in | |
| Section 3.2 of the Gemma Terms of Use, reproduced in the LICENSE file in this | |
| repository. Those restrictions apply to you, and you must pass them on to anyone | |
| you distribute this file or any derivative of it to. | |