Update README.md

README.md

@@ -10,24 +10,22 @@ pipeline_tag: feature-extraction
 
 # HTML-LM
 
-HTML-LM is a compact encoder model designed to generate general-purpose embeddings for HTML web pages, capturing both textual content and HTML structure. The embeddings can be used as inputs to lightweight downstream models for various classification and regression tasks such as explicit-content detection, shop-page classification, and spam-level prediction.
+HTML-LM is a compact encoder model designed to generate general-purpose embeddings for HTML web pages, capturing both textual content and HTML structure. The embeddings can be used as inputs to lightweight downstream models for various classification and regression tasks.
 
-The resulting representations are reusable across a wide range of downstream applications, including:
-
-* **Explicit content classification** — determine whether a page contains explicit or adult content.
-* **Article-page detection** — identify whether a webpage is primarily an article or editorial content.
-* **Product-page detection** — determine whether a page represents a product or e-commerce listing.
-* **Page clustering** — group similar webpages based on their content and structure.
-* **Page-quality regression** — estimate the overall quality and usefulness of a webpage.
-* **Web-spam detection** — estimate the level of spam or low-quality content on a page.
-* **Other** — HTML-LM representations can be reused for many other web-document understanding tasks, including **classification, regression, clustering, and other downstream applications**.
+HTML-LM representations can be reused across a **wide range of downstream applications**, supporting tasks such as **classification, regression, clustering, and broader web-document understanding**. Specifically, we use them for:
 
+* **Explicit content classification** — determining whether a webpage contains explicit or adult content.
+* **Article-page detection** — identifying whether a webpage primarily contains article or editorial content.
+* **Product-page detection** — determining whether a webpage represents a product or e-commerce listing.
+* **Page clustering** — grouping webpages with similar content and structural characteristics.
+* **Page-quality regression** — estimating the overall quality and usefulness of a webpage.
+* **Web-spam detection** — estimating the degree of spam or low-quality content on a webpage.
 
 ## Model details
 
 | Property        | Value          |
 | --------------- | -------------- |
-| Architecture    | ModernBERT     |
+| Architecture    | [ModernBERT](https://huggingface.co/answerdotai/ModernBERT-base) |
 | Parameters      | 154M           |
 | Hidden size     | 768            |
 | Layers          | 22             |
@@ -39,7 +37,7 @@ The model was trained from scratch for one pass over the training corpus using a
 
 * Masked Language Modeling (MLM)
 * Bag-of-Words prediction from `[CLS]`
-* Contrastive distillation from Qwen3-Embedding-8B and SeLLMa 8B
+* Contrastive distillation from [Qwen3-Embedding-8B](https://huggingface.co/Qwen/Qwen3-Embedding-8B) and [Seznam SeLLMa 8B](https://blog.seznam.cz/2024/10/diana-hlavacova-sellma-aneb-jak-v-seznamu-krotime-drave-jazykove-modely/)
 
 ## Training data
 
@@ -54,7 +52,7 @@ The corpus contains approximately:
 
 ## HTML preprocessing
 
-HTML-LM expects **preprocessed HTML**, rather than arbitrary raw HTML.
+> ⚠️ HTML-LM expects HTML documents to be preprocessed in a specific way, which is handled by the bundled AutoProcessor.from_pretrained("Seznam/html-lm").
 
 The included `HTMLLMProcessor` performs the following preprocessing steps:
 
@@ -65,10 +63,7 @@ The included `HTMLLMProcessor` performs the following preprocessing steps:
 5. Simplifying the DOM hierarchy.
 6. Normalizing whitespace.
 
-The same preprocessing approach was used to create the training data.
-
-> ⚠️ **Performance note:** The bundled processor may be slow out of the box for high-throughput or large-scale workloads. For improved performance, we recommend using it with `torch.utils.data.DataLoader` and setting `num_workers > 1`.
-
+> **Performance note:** The bundled processor may be slow out of the box for high-throughput or large-scale workloads. For improved performance, we recommend using it with `torch.utils.data.DataLoader` and setting `num_workers > 1`. See the example below.
 
 ## Usage
 
@@ -82,10 +77,12 @@ from transformers import AutoModel, AutoProcessor
 
 model_id = "Seznam/html-lm"
 
+# <class 'transformers_modules.hf.processor.HTMLLMProcessor'>
 processor = AutoProcessor.from_pretrained(
     model_id,
     trust_remote_code=True,
 )
+# <class 'transformers.models.modernbert.modeling_modernbert.ModernBertModel'>
 model = AutoModel.from_pretrained(model_id).to("cuda")
 model.eval()
 
@@ -112,6 +109,12 @@ document_embeddings = outputs.last_hidden_state[:, 0, :]
 
 print(document_embeddings.shape)
 # torch.Size([1, 768])
+
+print(document_embeddings)
+# tensor([[-2.0882e-01,  9.9287e-01, -1.0417e+00,  9.4297e-01, -8.4005e-01,
+#          1.1509e+00,  8.4631e-01, -3.2688e-01,  7.1889e-01,  3.8875e-02,
+#          ... 
+#          -2.9921e-01, -1.0583e+00,  1.4555e+00]], device='cuda:0')
 ```
 
 ## Performance
@@ -140,11 +143,24 @@ from transformers import AutoProcessor
 
 model_id = "Seznam/html-lm"
 
+# <class 'transformers_modules.hf.processor.HTMLLMProcessor'>
 processor = AutoProcessor.from_pretrained(
     model_id,
     trust_remote_code=True,
 )
-processor.preprocess_html("<html><body><h1>Example</h1><p>Some article text.</p></body></html>")
+
+html = """
+<html>
+    <body>
+        <div></div>
+        <h1 class='big'>Example</h1>
+        <p class='small'>Some article text.</p>
+    </body>
+</html>
+"""
+
+print(processor.preprocess_html(html))
+# <html><body><h1>Example</h1><p>Some article text.</p></body></html>
 ```
 
 ### Faster Preprocessing
@@ -255,4 +271,4 @@ If you use HTML-LM, please cite:
 
 ## Acknowledgements
 
-HTML-LM was developed by the **Seznam.cz Research team** as part of the HTML-LM project.
+HTML-LM was developed by the **Seznam.cz Research team** as part of the HTML-LM project.