Add model card, LICENSE (dots.ocr Agreement), and .gitattributes
Browse files- .gitattributes +3 -34
- LICENSE +118 -0
- README.md +213 -0
.gitattributes
CHANGED
|
@@ -1,35 +1,4 @@
|
|
| 1 |
-
*.
|
| 2 |
-
*.
|
| 3 |
-
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
-
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
-
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
-
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
-
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
-
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
-
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
-
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
-
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
-
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
-
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
-
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
-
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
-
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
-
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
-
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
-
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
-
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
-
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
-
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
-
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
-
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
-
|
| 27 |
-
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
-
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
-
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
-
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
-
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
-
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
-
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
-
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
-
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 1 |
+
*.hfq filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.mq* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
LICENSE
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
dots.ocr LICENSE AGREEMENT
|
| 2 |
+
|
| 3 |
+
Effective Date: [ August 8, 2025]
|
| 4 |
+
|
| 5 |
+
Copyright Holder: [Xingyin Information Technology (Shanghai) Co., Ltd]
|
| 6 |
+
|
| 7 |
+
This License Agreement (βAgreementβ) governs Your use, reproduction, modification, and distribution of dots.ocr (the "Model Materials"). This Agreement is designed to maximize the openness and use of the Model Materials while addressing the unique legal, ethical, and technical challenges posed by large language models.
|
| 8 |
+
|
| 9 |
+
WHEREAS, Licensor has developed the dots.ocr document parsing model and intends to distribute the Model Materials under an openβsource framework;
|
| 10 |
+
WHEREAS, traditional open-source licenses (e.g., the MIT License) may not fully address the complexity inherent complexities of document parsing models, namely their multiple components (code, weights, training data), potential ethical risks, dataβgovernance issues, and intellectualβproperty and liability questions regarding AIβgenerated content;
|
| 11 |
+
WHEREAS, Licensor seeks to provide a legal framework that ensures maximum access to and use of the Model Materials while clearly defining the rights, obligations, and liabilities of Licensee;
|
| 12 |
+
|
| 13 |
+
THEREFORE, the parties agree that, subject to the MIT License, they shall be bound by the following terms and conditions:
|
| 14 |
+
|
| 15 |
+
1. Definitions and Interpretation
|
| 16 |
+
Purpose: To define key terms used in this Agreement, particularly "Model Materials," ensuring clarity of the license scope beyond traditional software code. To clarify the order of precedence between this Agreement and the MIT License to avoid conflict.
|
| 17 |
+
|
| 18 |
+
1.1 βLicensorβ shall mean the entity providing the Model Materials under this Agreement, namely [Xingyin Information Technology (Shanghai) Co., Ltd].
|
| 19 |
+
|
| 20 |
+
1.2 βLicenseeβ or "You" shall mean any individual or entity exercising permissions granted by this Agreement.
|
| 21 |
+
|
| 22 |
+
1.3 βModel Materialsβ shall mean all materials provided by Licensor under this Agreement, including but not limited to:
|
| 23 |
+
(a) one or more machineβlearning models, including architecture and trained parameters (i.e., model weights);
|
| 24 |
+
(b) all associated preprocessing, training, inference, and fineβtuning code;
|
| 25 |
+
(c) training datasets and evaluation scripts (or their detailed descriptions and access mechanisms); and
|
| 26 |
+
(d) any accompanying documentation, metadata, and tools.
|
| 27 |
+
The above Model Materials shall be subject to the content published on the Licensorβs website or GitHub repository at https://github.com/rednote-hilab/dots.ocr.
|
| 28 |
+
|
| 29 |
+
1.4 βOutputsβ shall mean any content generated through the use of the Model Materials, such as text, tables, code,layout information, and formulas extracted from documents.
|
| 30 |
+
|
| 31 |
+
1.5 βMIT Licenseβ shall mean The MIT Open Source License published by the Massachusetts Institute of Technology.
|
| 32 |
+
|
| 33 |
+
1.6 Priority of Agreement. In the event of any conflict or inconsistency between this Agreement and the MIT License, the terms of the MIT License shall prevail. However, if the terms of the MIT License are ambiguous or silent on a particular matter, the provisions of this Agreement shall apply and supplement the MIT License.
|
| 34 |
+
|
| 35 |
+
2. Grant of Rights and Scope of Use
|
| 36 |
+
|
| 37 |
+
Purpose: To grant broad, permissive rights to the Licensee for the Model Materialsβincluding code, weights, data, and documentationβto ensure maximum openness and flexibility while clarifying the free use of model-generated content. Additionally, it clarifies the feasibility of transitioning from open-source to commercialβuse and the use of OpenAPI interfaces.
|
| 38 |
+
|
| 39 |
+
2.1 Grant of Copyright License. Subject to Licensee's compliance with this Agreement, Licensor hereby grants Licensee a perpetual, worldwide, nonβexclusive, noβcharge, royaltyβfree copyright license to use (run or test), reproduce, modify, create derivative works of, merge, publish, distribute the Model Materials; sublicense and/or sell copies of the Model Materials or any derivative works thereof; and incorporate the unmodified or modified Model Materials into proprietary products or services, including for commercial purposes, softwareβasβaβservice (SaaS) offerings, or via OpenAPI or other interfaces.
|
| 40 |
+
|
| 41 |
+
2.2 Fundamental Capabilities. The Model Materials only provide the fundamental modelβs capabilities. Licensees may develop derivative AI applications or undertake taskβspecific training thereon.
|
| 42 |
+
|
| 43 |
+
2.3 From Open Source to Commercial Use. The open-source release does not preclude Licensorβs commercial exploitation of the Model Materials, in whole or in part. Any such commercial use shall, at that time, be subject to license agreements between Licensor and applicable users.
|
| 44 |
+
|
| 45 |
+
2.4 APIβService Exception. Licensees who access the Model Materials through API calls or provide model services via API interfaces(without directly distributing model weights )shall not be subject to this Agreement unless otherwise expressly agreed. Instead, such use shall be governed by the API terms of use published by Licensor (if any).
|
| 46 |
+
|
| 47 |
+
3. Acceptable Use Policy and Prohibited Uses
|
| 48 |
+
|
| 49 |
+
3.1 Responsible Use. Licensee must use the Model Materials in a responsible, ethical, and lawful manner, in compliance with all applicable laws, regulations, industry standards, and best practices.
|
| 50 |
+
|
| 51 |
+
3.2 Enterprise OnβPremises Deployment. The Licensee may deploy the Model Materials in closedβsource, onβpremises enterprise environments.
|
| 52 |
+
|
| 53 |
+
3.3 Prohibited Uses. Any breach of the prohibitions below will result in the automatic termination of all licenses granted under this Agreement. Licensee agrees not to use the Model Materials or any derivative works thereof, in connection with:
|
| 54 |
+
(a) Identification and Utilization of Illegal/Harmful Content:Includes identifying graphic/text materials used for counterfeiting certificates/invoices, perpetrating fraud, or launching cyberattacks; or processing images containing illegal content such as violence, criminal activities, disinformation, or child exploitation.
|
| 55 |
+
(b) Privacy Infringement and Discriminatory Practices:Extracting personal sensitive information (e.g., ID numbers, medical records, biometric data) or protected characteristics (e.g., race, gender) from images without legal authorization or consent, for purposes of privacy violation, automated discriminatory decision-making, or harassment.
|
| 56 |
+
(c) Copyright Restrictions:Licensees shall not use the tool for unauthorized digitization of publications/document scanning or bulk scraping of content. Any use involving publications or other copyright-protected materials must first obtain relevant permissions.
|
| 57 |
+
|
| 58 |
+
4. Intellectual Property Ownership and Contributions
|
| 59 |
+
|
| 60 |
+
4.1 Licensor's Copyright Reservation. Licensor reserves all right, title, and interest in and to the Model Materials (including the model architecture, parameters, code, and original training data), except as expressly licensed herein. The original copyright of the Model Materials belongs to the Licensor.
|
| 61 |
+
|
| 62 |
+
4.2 Patent License. Subject to the terms and conditions of this Agreement, Licensor hereby grants Licensee a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Model Materials, where such license applies only to those patent claims licensable by the Lisensor that are necessarily infringed by its contribution(s).
|
| 63 |
+
If Licensee institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Model Materials constitute direct or contributory patent infringement, then any patent licenses granted under this License for the Model Materials shall terminate as of the date such litigation is asserted or filed.
|
| 64 |
+
|
| 65 |
+
4.3 Outputs: The Outputs generated through the use of the Model Materials generally refer to text, tables, layouts, and other content extracted from documents or images. The extracted content itself does not generate new intellectual property rights, and all intellectual property remains with the original authors or copyright holders. The Licensee is responsible for due diligence regarding the legality of the Outputs, particularly where the content extracted by the OCR model may be substantially similar to existing copyrighted works, which could present intellectual property infringement risks. The Licensor assumes no liability for such infringements.
|
| 66 |
+
4.4 Trademarks. Nothing in this License permits Licensee to make use of Licensorβs trademarks, trade names, logos (e.g., βrednote,β βXiaohongshu,β βdots.ocrβ) or to otherwise suggest endorsement or misrepresent the relationship between the parties, unless Licensorβs prior written approval is granted.
|
| 67 |
+
|
| 68 |
+
5. Data Governance, Privacy, and Security
|
| 69 |
+
|
| 70 |
+
5.1 Data Quality and Bias. Licensee shall use training data from lawful sources and is encouraged to conduct due diligence before deploying the Model Materials and to take reasonable steps to mitigate any known biases in its training data or applications.
|
| 71 |
+
|
| 72 |
+
5.2 Privacy Protection.
|
| 73 |
+
(a) SensitiveβData Restrictions. It is prohibited to use the Model Materials to process,or extract infer sensitive personal data protected under specific laws (such as GDPR or HIPAA), particularly when dealing with documents containing personally identifiable information (such as ID numbers, health data, financial information, etc.), unless Licensee has obtained all necessary consents, lawful basis, or authorizations, and has implemented adequate anonymization, pseudonymization, or other privacy-enhancing technologies.
|
| 74 |
+
(b) Data Minimization and Purpose Limitation. The Licensee shall follow the principle of data minimization when using the OCR Model, processing only the user data necessary for specific, explicit, and lawful purposes. Specifically, the OCR Model should avoid processing unnecessary sensitive data and ensure compliance with applicable privacy protection laws during data handling.
|
| 75 |
+
(c) Transparency. Licensee shall provide clear and transparent privacy policies and terms of use when processing user data, particularly during document scanning and information extraction. .
|
| 76 |
+
|
| 77 |
+
5.3 Security Measures. Licensee shall implement appropriate technical and administrative safeguards to protect the Model Materials and any associated data against unauthorized access, disclosure, alteration, or destruction. Such measures may include, but are not limited to, encryption, access controls, logging, and audit trails.
|
| 78 |
+
|
| 79 |
+
5.4 Further Training. Licensee may only use userβprovided input or Outputs for training, fine-tuning, or improving other AI models if it has obtained the specific and informed consent of data subjects.
|
| 80 |
+
|
| 81 |
+
6. Disclaimer of Warranty and Limitation of Liability
|
| 82 |
+
|
| 83 |
+
6.1 βAS ISβ Basis. Unless required by applicable law, the Model Materials are provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. Licensee is solely responsible for determining the appropriateness of using or redistributing the Model Materials and assume any risks associated with the exercise of permissions under this License. Licensor does not provide any warranty of non-infringement but represents that no infringing code has been knowingly included.
|
| 84 |
+
|
| 85 |
+
6.2 Outputs Disclaimer. As a neutral technology, Licensor disclaims all liability for the accuracy, completeness, reliability, safety, legality, or suitability of any Outputs. The Licensee is solely responsible for verifying the accuracy and appropriateness of AI-generated content and shall provide appropriate disclosures when publishing or relying upon such content.
|
| 86 |
+
|
| 87 |
+
6.3 Limitation of Liability and Recourse. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, shall Licensor or contributors be liable for any claims, damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Model Materials (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Licensor has been advised of the possibility of such damages. If such losses are incurred, recourse may be sought against the Licensee responsible for causing the loss.
|
| 88 |
+
|
| 89 |
+
6.4 ContentβFiltering Disclaimer. Although the Model Materials may include contentβfiltering mechanisms, Licensor makes no warranties of any kind regarding the stability, quality, accuracy, completeness, or any specific outcome of Outputs. Licensee is solely responsible for reviewing, verifying, and performing quality control on Outputs and assumes all associated risks and liabilities.
|
| 90 |
+
|
| 91 |
+
7. Attribution and License Reservation
|
| 92 |
+
|
| 93 |
+
7.1 License. When distributing or redistributing the Model Materials, Licensee must give any other recipients of the Model Materials a copy of this Agreement.
|
| 94 |
+
|
| 95 |
+
7.2 Copyright and Notices. When distributing any part of the Model Materials, Licensee must retain all copyright, patent, trademark, and attribution notices included in the Model Materials.
|
| 96 |
+
|
| 97 |
+
7.3 Attribution. Licensee is encouraged to prominently display the name of Licensor and the Model Materials in any public statements, products, or services that contain the Model Materials (or any derivative works thereof), to promote transparency and community trust. If Licensee distributes modified weights or fineβtuned models based on the Model Materials, Licensee must prominently display the following statement in the related website or documentation: βBuilt with dots.ocr.β
|
| 98 |
+
|
| 99 |
+
8. Governing Law and Dispute Resolution
|
| 100 |
+
|
| 101 |
+
8.1 Governing Law. This Agreement shall be governed by and construed in accordance with the laws of the Peopleβs Republic of China, without regard to its conflict of laws principles.
|
| 102 |
+
|
| 103 |
+
8.2 Dispute Resolution. Any dispute claim, or disagreement arising out of or relating to this Agreement shall first be resolved through amicable consultation. If such consultation fails, the dispute shall be submitted to the Hangzhou Arbitration Commission for arbitration. The arbitration shall be conducted in accordance with the laws of China, and the place of arbitration shall be [Hangzhou, China]. The arbitral award shall be final and binding upon both parties.
|
| 104 |
+
|
| 105 |
+
9. Regulatory Compliance Amendments
|
| 106 |
+
In the event that any part of this Agreement becomes invalid or requires adjustment due to changes in applicable laws or regulations, Licensor reserves the right to issue a revised version of this Agreement. Licensee shall migrate to the new version within [e.g., ninety (90)] days of its release; otherwise, all rights granted under this Agreement shall automatically terminate.
|
| 107 |
+
|
| 108 |
+
10. Security Reporting
|
| 109 |
+
Licensee discovering any security vulnerability in the Model Materials may report it to Licensor via: dots-feedback@xiaohongshu.com. Licensee shall not disclose vulnerability details until Licensor issues an official remediation, unless otherwise required by law.
|
| 110 |
+
|
| 111 |
+
-----
|
| 112 |
+
|
| 113 |
+
This copy of the dots.ocr License Agreement is included verbatim, as required by
|
| 114 |
+
the Agreement Β§7.1, from the upstream model repository
|
| 115 |
+
https://huggingface.co/rednote-hilab/dots.ocr (file "dots.ocr LICENSE AGREEMENT").
|
| 116 |
+
The model weights redistributed in this repository are a quantized derivative
|
| 117 |
+
work; the Agreement is MIT-based and, per Β§1.6, the terms of the MIT License
|
| 118 |
+
prevail where the two conflict.
|
README.md
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
base_model: rednote-hilab/dots.ocr
|
| 4 |
+
base_model_relation: quantized
|
| 5 |
+
library_name: hipfire
|
| 6 |
+
pipeline_tag: image-text-to-text
|
| 7 |
+
language:
|
| 8 |
+
- en
|
| 9 |
+
- zh
|
| 10 |
+
- multilingual
|
| 11 |
+
tags:
|
| 12 |
+
- hipfire
|
| 13 |
+
- quantized
|
| 14 |
+
- ocr
|
| 15 |
+
- document-parse
|
| 16 |
+
- layout
|
| 17 |
+
- table
|
| 18 |
+
- formula
|
| 19 |
+
- vlm
|
| 20 |
+
- qwen2-vl
|
| 21 |
+
- dots.ocr
|
| 22 |
+
---
|
| 23 |
+
|
| 24 |
+
# hipfire-dots.ocr
|
| 25 |
+
|
| 26 |
+
> **License:** MIT, under the upstream
|
| 27 |
+
> [dots.ocr License Agreement](LICENSE) (copied verbatim from
|
| 28 |
+
> [rednote-hilab/dots.ocr](https://huggingface.co/rednote-hilab/dots.ocr)). The
|
| 29 |
+
> Agreement sits on top of the MIT license; where the two conflict, **MIT
|
| 30 |
+
> prevails** (Agreement Β§1.6). This is a quantized derivative of a model created
|
| 31 |
+
> and released by **rednote / Xiaohongshu**
|
| 32 |
+
> ([Xingyin Information Technology (Shanghai) Co., Ltd.](https://huggingface.co/rednote-hilab));
|
| 33 |
+
> all upstream attribution and license terms apply. **Built with dots.ocr.**
|
| 34 |
+
|
| 35 |
+
[hipfire](https://github.com/Kaden-Schutt/hipfire)-native **Q8** quantization of
|
| 36 |
+
[rednote-hilab/dots.ocr](https://huggingface.co/rednote-hilab/dots.ocr) β a
|
| 37 |
+
multilingual document **layout-parsing** vision-language model that unifies
|
| 38 |
+
layout detection and content recognition in a single VLM while preserving
|
| 39 |
+
reading order. Despite a compact 1.7B-parameter LLM foundation, dots.ocr is
|
| 40 |
+
state-of-the-art on [OmniDocBench](https://github.com/opendatalab/OmniDocBench).
|
| 41 |
+
|
| 42 |
+
The `.hfq` file runs with the [hipfire](https://github.com/Kaden-Schutt/hipfire)
|
| 43 |
+
inference engine β a Rust + HIP/ROCm-direct runtime for AMD RDNA GPUs with no
|
| 44 |
+
Python in the hot path (Ollama-style UX). It is hipfire's **HFQ container
|
| 45 |
+
format** (`HFQM`-magic manifest): **not** GGUF or safetensors, and it will **not**
|
| 46 |
+
load in `llama.cpp` or `transformers`. All weights are stored at **Q8** and
|
| 47 |
+
dequantized to f32 on the fly inside the GEMV kernels.
|
| 48 |
+
|
| 49 |
+
dots.ocr is a first-class supported architecture in hipfire's registry
|
| 50 |
+
("Qwen2-VL-family layout-extraction VLM β image β structured OCR"; bring-your-own
|
| 51 |
+
via `hipfire quantize`).
|
| 52 |
+
|
| 53 |
+
## Files
|
| 54 |
+
|
| 55 |
+
| File | Quant | Size | sha256 |
|
| 56 |
+
|------|-------|------|--------|
|
| 57 |
+
| `dots-ocr.q8.hfq` | Q8 (all tensors) | 4.42 GB | `eec256b1β¦b6f5b268` |
|
| 58 |
+
|
| 59 |
+
### HFQ header β embedded architecture / config
|
| 60 |
+
|
| 61 |
+
The `HFQM` manifest carries the full dots.ocr config, tokenizer, and generation
|
| 62 |
+
config verbatim from the upstream checkpoint:
|
| 63 |
+
|
| 64 |
+
- **architecture:** `dots_ocr` (`DotsOCRForCausalLM`) β Qwen2-VL family
|
| 65 |
+
- **LLM:** 28 layers, hidden 1536, 12 attention heads / 2 KV heads (GQA),
|
| 66 |
+
intermediate 8960, vocab 151936, RoPE ΞΈ = 1e6, max position 131072, SwiGLU
|
| 67 |
+
- **Vision tower:** 42 layers, embed_dim 1536, 12 heads, patch 14,
|
| 68 |
+
spatial-merge 2, 3-channel; optimal under ~11.3M px
|
| 69 |
+
- **generation:** `eos_token_id` = `[151643, 151673]`, `max_length` 32768
|
| 70 |
+
- **source dtype:** bfloat16 β re-quantized to Q8
|
| 71 |
+
|
| 72 |
+
## About dots.ocr
|
| 73 |
+
|
| 74 |
+
**dots.ocr** is a powerful, multilingual document parser that unifies layout
|
| 75 |
+
detection and content recognition within a single vision-language model while
|
| 76 |
+
maintaining good reading order. Despite its compact 1.7B-parameter LLM
|
| 77 |
+
foundation, it achieves state-of-the-art (SOTA) performance.
|
| 78 |
+
|
| 79 |
+
1. **Powerful Performance:** dots.ocr achieves SOTA for text, tables, and reading
|
| 80 |
+
order on OmniDocBench, while delivering formula recognition comparable to much
|
| 81 |
+
larger models like Doubao-1.5 and Gemini 2.5-Pro.
|
| 82 |
+
2. **Multilingual Support:** robust parsing for low-resource languages, with
|
| 83 |
+
decisive advantages across both layout detection and content recognition on
|
| 84 |
+
dots.ocr's in-house 100-language benchmark.
|
| 85 |
+
3. **Unified and Simple Architecture:** a single VLM is far more streamlined than
|
| 86 |
+
conventional multi-model pipelines. Switching tasks is just changing the input
|
| 87 |
+
prompt β competitive with dedicated detectors like DocLayout-YOLO.
|
| 88 |
+
4. **Efficient and Fast:** built on a compact 1.7B LLM, it is faster than larger
|
| 89 |
+
foundation-based parsers.
|
| 90 |
+
|
| 91 |
+
### Performance highlights (upstream dots.ocr, bfloat16)
|
| 92 |
+
|
| 93 |
+
> These are the upstream model's published numbers. See the
|
| 94 |
+
> [original card](https://huggingface.co/rednote-hilab/dots.ocr) and the
|
| 95 |
+
> [dots.ocr repo](https://github.com/rednote-hilab/dots.ocr) for the full
|
| 96 |
+
> benchmark tables (OmniDocBench, dots.ocr-bench, olmOCR-bench, layout
|
| 97 |
+
> detection).
|
| 98 |
+
|
| 99 |
+
**OmniDocBench β end-to-end** (Edit distance β is better; TEDS β is better).
|
| 100 |
+
Lower is better except Table TEDS.
|
| 101 |
+
|
| 102 |
+
| Method | Overall Edit β (EN / ZH) | Text Edit β (EN / ZH) | Table TEDS β (EN / ZH) | Read Order Edit β (EN / ZH) |
|
| 103 |
+
|---|---|---|---|---|
|
| 104 |
+
| MinerU 2 | 0.139 / 0.240 | 0.047 / 0.109 | 82.5 / 79.0 | 0.069 / 0.118 |
|
| 105 |
+
| MonkeyOCR-pro-3B | 0.138 / 0.206 | 0.067 / 0.107 | 81.5 / 87.5 | 0.100 / 0.185 |
|
| 106 |
+
| Mistral OCR | 0.268 / 0.439 | 0.072 / 0.325 | 75.8 / 63.6 | 0.083 / 0.284 |
|
| 107 |
+
| Gemini 2.5-Pro | 0.148 / 0.212 | 0.055 / 0.168 | 85.8 / 86.4 | 0.049 / 0.121 |
|
| 108 |
+
| doubao-1.5-thinking-vision-pro | 0.140 / 0.162 | 0.043 / 0.085 | 83.3 / **89.3** | 0.058 / 0.094 |
|
| 109 |
+
| **dots.ocr** | **0.125 / 0.160** | **0.032 / 0.066** | **88.6 / 89.0** | **0.040 / 0.067** |
|
| 110 |
+
|
| 111 |
+
dots.ocr also leads **olmOCR-bench** overall (**79.1 Β± 1.0**, best on Tables
|
| 112 |
+
88.3, Multi-column 82.4, Base 99.5) and **dots.ocr-bench** (100 languages,
|
| 113 |
+
Overall Edit **0.177**), and its layout-detection-only mode reaches
|
| 114 |
+
**F1@IoU .50 = 0.930** overall.
|
| 115 |
+
|
| 116 |
+
## Usage (hipfire)
|
| 117 |
+
|
| 118 |
+
```bash
|
| 119 |
+
# install hipfire (Linux + ROCm 6+)
|
| 120 |
+
curl -L https://raw.githubusercontent.com/Kaden-Schutt/hipfire/master/scripts/install.sh | bash
|
| 121 |
+
|
| 122 |
+
# download this Q8 file directly
|
| 123 |
+
hf download hipfire-models/hipfire-dots.ocr dots-ocr.q8.hfq --local-dir ~/.hipfire/models
|
| 124 |
+
|
| 125 |
+
# serve it (OpenAI-compatible HTTP API on 0.0.0.0:11435)
|
| 126 |
+
hipfire serve --model ~/.hipfire/models/dots-ocr.q8.hfq
|
| 127 |
+
|
| 128 |
+
# or one-shot an image
|
| 129 |
+
hipfire run --model dots-ocr.q8.hfq --image page.png --prompt "$(cat prompt.txt)"
|
| 130 |
+
```
|
| 131 |
+
|
| 132 |
+
### Layout-parsing prompt
|
| 133 |
+
|
| 134 |
+
dots.ocr is prompt-driven β the same VLM does full parse, detection-only,
|
| 135 |
+
text-only, or grounding-OCR depending on the input prompt. The default
|
| 136 |
+
"parse all" prompt (see the upstream
|
| 137 |
+
[prompts.py](https://github.com/rednote-hilab/dots.ocr/blob/master/dots_ocr/utils/prompts.py))
|
| 138 |
+
asks for one JSON object with each layout element's bbox, category, and text:
|
| 139 |
+
|
| 140 |
+
```
|
| 141 |
+
Please output the layout information from the PDF image, including each layout
|
| 142 |
+
element's bbox, its category, and the corresponding text content within the bbox.
|
| 143 |
+
|
| 144 |
+
1. Bbox format: [x1, y1, x2, y2]
|
| 145 |
+
2. Layout Categories: ['Caption','Footnote','Formula','List-item','Page-footer',
|
| 146 |
+
'Page-header','Picture','Section-header','Table','Text','Title'].
|
| 147 |
+
3. Text Extraction & Formatting Rules:
|
| 148 |
+
- Picture: omit the text field.
|
| 149 |
+
- Formula: format its text as LaTeX.
|
| 150 |
+
- Table: format its text as HTML.
|
| 151 |
+
- All Others (Text, Title, etc.): format their text as Markdown.
|
| 152 |
+
4. Constraints:
|
| 153 |
+
- The output text must be the original text from the image, with no translation.
|
| 154 |
+
- All layout elements must be sorted according to human reading order.
|
| 155 |
+
5. Final Output: The entire output must be a single JSON object.
|
| 156 |
+
```
|
| 157 |
+
|
| 158 |
+
Switch tasks by swapping the prompt: `prompt_layout_only_en` (detection only),
|
| 159 |
+
`prompt_ocr` (text only, skipping headers/footers), or `prompt_grounding_ocr`
|
| 160 |
+
(parse a single bbox).
|
| 161 |
+
|
| 162 |
+
## Quantization & validation
|
| 163 |
+
|
| 164 |
+
Quantized from the upstream bfloat16 safetensors to **Q8** (8-bit per-tensor)
|
| 165 |
+
through hipfire's `hipfire quantize` path; weights are stored at Q8 and
|
| 166 |
+
dequantized to f32 inside the GEMV kernels β the math is f32, the bit-width is a
|
| 167 |
+
storage/bandwidth dial. The upstream tokenizer, `chat_template`, config, and
|
| 168 |
+
generation config are embedded in the `HFQM` manifest.
|
| 169 |
+
|
| 170 |
+
architecture `dots_ocr` in the hipfire HFQ header.
|
| 171 |
+
|
| 172 |
+
## Limitations (from the upstream card)
|
| 173 |
+
|
| 174 |
+
- **Complex elements:** not yet perfect on high-complexity tables and formula
|
| 175 |
+
extraction; pictures in documents are currently not parsed.
|
| 176 |
+
- **Parsing failures:** may fail when the character-to-pixel ratio is very high
|
| 177 |
+
(enlarge the image, or raise PDF DPI β 200 is recommended; the model is optimal
|
| 178 |
+
under ~11.3M px). Runs of special characters (`...`, `___`) can cause output to
|
| 179 |
+
repeat β in that case switch to `prompt_layout_only_en`, `prompt_ocr`, or
|
| 180 |
+
`prompt_grounding_ocr`.
|
| 181 |
+
- **Throughput:** despite the 1.7B LLM, not yet optimized for high-throughput
|
| 182 |
+
bulk-PDF processing.
|
| 183 |
+
|
| 184 |
+
## Citation
|
| 185 |
+
|
| 186 |
+
If you use this work, please cite the original dots.ocr model and team:
|
| 187 |
+
|
| 188 |
+
```
|
| 189 |
+
@misc{dotsocr,
|
| 190 |
+
title = {dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model},
|
| 191 |
+
author = {rednote-hilab},
|
| 192 |
+
year = {2025},
|
| 193 |
+
url = {https://huggingface.co/rednote-hilab/dots.ocr}
|
| 194 |
+
}
|
| 195 |
+
```
|
| 196 |
+
|
| 197 |
+
## License & attribution
|
| 198 |
+
|
| 199 |
+
This repository distributes a **quantized Derivative Work** of
|
| 200 |
+
[rednote-hilab/dots.ocr](https://huggingface.co/rednote-hilab/dots.ocr). The
|
| 201 |
+
model is Β© **Xingyin Information Technology (Shanghai) Co., Ltd.** and is
|
| 202 |
+
licensed under the [dots.ocr License Agreement](LICENSE) (MIT-based; copied
|
| 203 |
+
verbatim into this repo from the upstream repository).
|
| 204 |
+
|
| 205 |
+
**Modification notice:** the upstream bfloat16 safetensors weights were
|
| 206 |
+
re-quantized into hipfire's Q8 / HFQ container format, and the upstream
|
| 207 |
+
tokenizer / chat template / config were embedded into the HFQ metadata. No other
|
| 208 |
+
changes to the model. Per the Agreement Β§7, redistribution includes a copy of the
|
| 209 |
+
Agreement and retains all attribution notices; modified-weight releases should
|
| 210 |
+
display **"Built with dots.ocr."**
|
| 211 |
+
|
| 212 |
+
hipfire itself is dual-licensed MIT / Apache-2.0 β see the
|
| 213 |
+
[hipfire repo](https://github.com/Kaden-Schutt/hipfire).
|