urnova-95m / DATA_AND_THIRD_PARTY_NOTICE.md
mahwizzzz's picture
Add weights licence and HPLT provenance notices
d36bc6c verified
|
Raw
History Blame Contribute Delete
3.3 kB

Data and Third-Party Notice

Scope

The urnova-95m model was trained from scratch. It was not initialized from mBERT, XLM-R, DunbaaBERT, HPLT-BERT Urdu, or another third-party model checkpoint.

The original urnova model weights and locally generated tokenizer artifacts are distributed under the Apache License, Version 2.0, to the extent of the rights held by the urnova publisher.

This licence does not relicense the underlying pretraining documents.

Training-data source

The production corpus was derived from the Urdu Arabic-script portion of HPLT 3.0.

HPLT states that:

  • it does not own the text from which the dataset was extracted;
  • it licenses the packaging of the dataset under Creative Commons CC0;
  • users remain responsible for compliance with applicable law.

Official dataset identifier:

HPLT/HPLT3.0

No raw or reconstructed HPLT documents are included in this model repository.

Recorded production provenance

The frozen production lineage records:

  • source documents: 1,921,027;
  • source-token count: 1,246,973,570;
  • production training tokens processed: approximately 4.0 billion;
  • source corpus name in the deduplication manifest: hplt3;
  • source file SHA-256: b4ede7d4548fc3db385666eddd796ae2ceb0d37788bbe85a13ff8abd81ebb991;
  • tokenized output SHA-256: b07ad6d48efe3e5147d875b09acf4a53ce675d078e3d868e8d778e15c8c863aa;
  • upstream corpus identity SHA-256: 0ba4bf10ad09c3b8105f8ac20dd62953e8537664b8cda0ea4557a6ec275817d3;
  • tokenizer vocabulary SHA-256: 3acb94ea6bca8b2a0256d9d46c545bb9c033977c71671f8bc52cb1aa6ada92bc.

Phases A, B, and C used the same tokenized production corpus with different packing and context-length policies.

Provenance limitations

The local production records do not preserve an immutable HPLT upstream revision or acquisition timestamp.

The tokenizer-training manifest records 199,117 documents from the local file dedup-200k.jsonl, with SHA-256:

75d73d7d874525ab9c076f62ebbeb6bfd7bd0b63acac68bee2db657771855f34

That tokenizer manifest does not independently record the original dataset identifier or immutable upstream revision.

These are known provenance limitations and must not be represented as fully resolved source-level provenance.

Third-party rights

Apache-2.0 applies only to rights held by the licensors in the distributed urnova artifacts. It does not grant permission to use:

  • copyrighted text that may be reproduced from training data;
  • names, logos, or trademarks appearing in source material;
  • personal information or content subject to privacy or publicity rights;
  • material restricted by website, publisher, contractual, or database terms.

Users are responsible for determining whether their intended use complies with applicable law.

Takedown and concerns

For concerns relating to this model repository, open a discussion at:

https://huggingface.co/mahwizzzz/urnova-95m/discussions

Provide enough information to identify the affected material and explain the basis of the request.

Valid requests will be investigated. Where appropriate, affected sources will be excluded from future training runs, and the model card will identify which previous model versions remain affected.

HPLT publishes its own dataset notice-and-takedown procedure separately.