realAABeigi/ZAT-E1

Model Description

• Model Type: Sentence Transformer
• Base Model: sentence-transformers/all-distilroberta-v1
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

---

Semantic vs. Logical Embedding: Why ZAT-E1 stands out?

Most embedding models on the market suffer from the "Lexical Overlap Trap". They assign high similarity scores to sentences that share the same words, even if their logical meanings are completely contradictory. ZAT-E1 is heavily optimized to understand logical reasoning (Entailment vs. Contradiction).

Case Study: Contradiction Detection Look at how ZAT-E1 compares to Gemini-embedding-001:

• S1: "I forgot to charge my laptop last night."
• S2: "The battery died in the middle of my presentation." (Logical Entailment)
• S3: "I remembered to charge my laptop last night." (Logical Contradiction)

Model	S1 vs S2 (Entailment)	S1 vs S3 (Contradiction)
Gemini-embedding-001	0.8458	0.9152 (Failed: Ignored logical negation)
ZAT-E1	0.8937	-0.2065 (Success: Detected contradiction)

Stress Testing on FOLIO (First-Order Logic Inference Dataset)

To push ZAT-E1 to its limits, we evaluated it on the FOLIO dataset, a benchmark strictly designed for testing First-Order Logic reasoning in Large Language Models (LLMs like GPT-4).

• ZAT-E1 Zero-Shot Overall Accuracy: 42.86% (For context, early un-tuned LLMs like GPT-3 (davinci) scored ~40-45% on this exact benchmark).

While an embedding model is not a reasoning engine, ZAT-E1's ability to achieve above-random logical differentiation (particularly its strong bias against logical contradictions) makes it exceptionally robust for RAG pipelines, ensuring that retrieved documents are logically aligned with the user's query, not just lexically similar.

---

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("realAABeigi/ZAT-E1")
# Run inference
sentences = [
          "I forgot to charge my laptop last night."
 ‌‌  ‌‌       "The battery died in the middle of my presentation." (Logical Entailment)
          "I remembered to charge my laptop last night." (Logical Contradiction)

]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.9199, -0.1705],
#         [ 0.9199,  1.0000, -0.0191],
#         [-0.1705, -0.0191,  1.0000]])