Upload folder using huggingface_hub

README.md

@@ -6,8 +6,9 @@ tags:
 - civil-engineering
 - technical-expert
 - structural-analysis
-- TinyLlama/TinyLlama-1.1B-Chat-v1.0
-- microsoft/phi-1_5
+- rag-verification
+- EleutherAI/pythia-410m
+- EleutherAI/pythia-1b
 - EleutherAI/pythia-1.4b
 ---
 
@@ -16,10 +17,46 @@ tags:
 shivik-civil-engineering-expert is a specialized language model designed for civil engineering professionals. It combines multiple models using [mergekit](https://github.com/cg123/mergekit) to enhance technical expertise in structural analysis, materials science, construction management, and related disciplines.
 
 ## Models Merged
-* [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
-* [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5)
+* [EleutherAI/pythia-410m](https://huggingface.co/EleutherAI/pythia-410m)
+* [EleutherAI/pythia-1b](https://huggingface.co/EleutherAI/pythia-1b)
 * [EleutherAI/pythia-1.4b](https://huggingface.co/EleutherAI/pythia-1.4b)
 
+## Technical Details
+
+This model is created by merging models from the Pythia family, which ensures architectural compatibility while combining different knowledge capabilities:
+- The base Pythia-410m model provides foundation capabilities
+- The Pythia-1b model adds deeper reasoning
+- The Pythia-1.4b model contributes more specialized knowledge
+
+## RAG Verification System
+
+This model includes a specialized civil engineering RAG verification system that:
+- Verifies technical claims against retrieved engineering documents
+- Checks engineering units and values for accuracy
+- Identifies potentially incorrect specifications
+- Ensures compliance with engineering standards and codes
+
+### Using the RAG Verifier
+
+```python
+from civil_engineering_rag_verifier import CivilEngineeringRAGVerifier
+
+# Initialize the verifier
+verifier = CivilEngineeringRAGVerifier()
+
+# Verify model responses against retrieved documents
+verified, modified_response, details = verifier.verify_response(
+    model_response="Your engineering text here...",
+    retrieved_documents=[{"content": "Reference document content..."}]
+)
+
+# Check verification results
+print(f"Response verified: {verified}")
+print(f"Score: {details['verification_score']}")
+```
+
+See the included `civil_engineering_example.py` for a complete usage example.
+
 ## Target Applications
 
 This model is optimized for civil engineering applications, including:
@@ -35,26 +72,28 @@ This model is optimized for civil engineering applications, including:
 ```yaml
 slices:
   - sources:
-      - model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
-        layer_range: [0, 24]
-        # First expert - open access base model
-      - model: microsoft/phi-1_5
-        layer_range: [0, 24]
-        # Second expert - open access model
+      - model: EleutherAI/pythia-410m
+        layer_range: [0, 16]
+        # Small model from Pythia family
+      - model: EleutherAI/pythia-1b
+        layer_range: [0, 16]
+        # Medium model from Pythia family
       - model: EleutherAI/pythia-1.4b
-        layer_range: [0, 24]
-        # Third expert - another open access model
+        layer_range: [0, 16]
+        # Larger model from Pythia family
 merge_method: ties
-base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
+base_model: EleutherAI/pythia-410m
 parameters:
-  density: 0.5  # Density parameter for knowledge retention
+  density: 0.5  # Higher density for more comprehensive technical knowledge
   weights:
-    - 1.0  # Weight for TinyLlama
-    - 0.9  # Weight for Phi-1.5
-    - 0.8  # Weight for Pythia
+    - 0.8  # Weight for Pythia-410m
+    - 0.9  # Weight for Pythia-1b
+    - 1.0  # Weight for Pythia-1.4b
 dtype: bfloat16
 ```
 
 ## Usage Notes
 
-This model demonstrates enhanced capabilities in technical reasoning and engineering-specific knowledge compared to general language models. It is particularly suitable for professional civil engineers seeking AI assistance with technical documentation, calculations, and engineering problem-solving.
+This model demonstrates enhanced capabilities in technical reasoning and engineering-specific knowledge compared to general language models. It is particularly suitable for professional civil engineers seeking AI assistance with technical documentation, calculations, and engineering problem-solving.
+
+The RAG verification system helps ensure that model outputs align with accepted engineering standards and practices, providing an additional layer of confidence for critical engineering applications.

civil_engineering_example.py

@@ -0,0 +1,70 @@
+
+from civil_engineering_rag_verifier import CivilEngineeringRAGVerifier
+
+def main():
+    # Initialize the verifier
+    verifier = CivilEngineeringRAGVerifier(similarity_threshold=0.7)
+    
+    # Example model response for a structural engineering question
+    model_response = """
+    For a reinforced concrete beam with a span of 6 m carrying a uniform load of 25 kN/m:
+    
+    1. The maximum bending moment will be approximately 112.5 kNm at midspan.
+    2. Assuming a concrete strength of 30 MPa and steel yield strength of 500 MPa, a beam depth of 450 mm would be appropriate.
+    3. The minimum reinforcement ratio should be 0.3% according to ACI 318.
+    4. The beam should have a minimum width of 250 mm to accommodate the required reinforcement.
+    5. Shear reinforcement with 10 mm diameter stirrups at 200 mm spacing will be sufficient near the supports.
+    """
+    
+    # Example retrieved engineering documents
+    retrieved_documents = [
+        {
+            "content": "For simply supported beams with uniform loading (w), the maximum bending moment occurs at midspan and equals wL²/8, where L is the span length.",
+            "source": "structural_engineering_handbook_p127"
+        },
+        {
+            "content": "ACI 318 specifies a minimum flexural reinforcement ratio of 0.33% for beams using Grade 60 (420 MPa) steel. For higher strength steels, this may be adjusted.",
+            "source": "concrete_design_manual"
+        },
+        {
+            "content": "For concrete with compressive strength of 30 MPa and steel with yield strength of 500 MPa, recommended beam depths typically range from L/12 to L/15 for normal loading conditions.",
+            "source": "design_guidelines"
+        },
+        {
+            "content": "Shear reinforcement spacing should not exceed d/2 or 600 mm, whichever is smaller, where d is the effective depth of the beam.",
+            "source": "code_provisions"
+        }
+    ]
+    
+    # Verify the response
+    is_verified, modified_response, verification_details = verifier.verify_response(
+        model_response, 
+        retrieved_documents
+    )
+    
+    # Print results
+    print(f"Response verified: {is_verified}")
+    print(f"Verification score: {verification_details['verification_score']:.2f}")
+    print(f"Claims verified: {verification_details['claims_verified']}/{verification_details['total_claims']}")
+    print(f"Units verified: {verification_details['units_verified']}/{verification_details['total_units']}")
+    
+    print("\nOriginal response:")
+    print(model_response)
+    
+    if not is_verified:
+        print("\nModified response with verification:")
+        print(modified_response)
+    
+    print("\nDetailed verification results:")
+    print("\nTechnical Claims:")
+    for claim_result in verification_details['claim_details']:
+        status = "✓" if claim_result['verified'] else "✗"
+        print(f"{status} {claim_result['claim']}")
+    
+    print("\nUnits and Values:")
+    for unit_result in verification_details['unit_details']:
+        status = "✓" if unit_result['verified'] else "✗"
+        print(f"{status} {unit_result['unit_value']}")
+
+if __name__ == "__main__":
+    main()

civil_engineering_rag_verifier.py

@@ -0,0 +1,254 @@
+
+# Civil Engineering RAG Verification Module
+import re
+from typing import List, Dict, Any, Tuple
+
+class CivilEngineeringRAGVerifier:
+    """
+    A verification module specialized for civil engineering applications.
+    Ensures outputs are consistent with engineering documents and standards.
+    """
+    
+    def __init__(self, similarity_threshold: float = 0.7):
+        """Initialize the RAG verifier with configurable threshold."""
+        self.similarity_threshold = similarity_threshold
+        self.engineering_units = {
+            "length": ["mm", "cm", "m", "km", "in", "ft", "yd", "mi"],
+            "area": ["mm²", "cm²", "m²", "km²", "ha", "in²", "ft²", "yd²", "acre"],
+            "volume": ["mm³", "cm³", "m³", "L", "in³", "ft³", "yd³", "gal"],
+            "force": ["N", "kN", "MN", "lbf", "kip"],
+            "pressure": ["Pa", "kPa", "MPa", "psi", "psf", "ksf"],
+            "density": ["kg/m³", "g/cm³", "lb/ft³"],
+            "temperature": ["°C", "°F", "K"],
+            "time": ["s", "min", "h", "day", "week", "month", "year"]
+        }
+        
+    def verify_response(self, 
+                        model_response: str, 
+                        retrieved_documents: List[Dict[str, Any]]) -> Tuple[bool, str, Dict]:
+        """
+        Verify if the model's response is consistent with civil engineering documents.
+        
+        Args:
+            model_response: The text response generated by the model
+            retrieved_documents: List of documents retrieved by the RAG system
+            
+        Returns:
+            Tuple containing:
+            - bool: Whether the response is verified
+            - str: Modified response if needed
+            - Dict: Verification details
+        """
+        # Combine all document content for verification
+        combined_context = "\n".join([doc.get("content", "") for doc in retrieved_documents])
+        
+        # Extract technical claims and specifications
+        claims = self._extract_technical_claims(model_response)
+        
+        # Extract units and numerical values
+        units_and_values = self._extract_units_and_values(model_response)
+        
+        # Verify each claim against the context
+        verification_results = []
+        for claim in claims:
+            is_supported = self._verify_technical_claim(claim, combined_context)
+            verification_results.append({
+                "claim": claim,
+                "verified": is_supported
+            })
+        
+        # Verify units and values
+        unit_verification_results = []
+        for unit_value in units_and_values:
+            is_correct = self._verify_unit_and_value(unit_value, combined_context)
+            unit_verification_results.append({
+                "unit_value": unit_value,
+                "verified": is_correct
+            })
+        
+        # Calculate overall verification score
+        verified_claims = sum(1 for v in verification_results if v["verified"])
+        verified_units = sum(1 for v in unit_verification_results if v["verified"])
+        
+        total_items = len(claims) + len(units_and_values)
+        if total_items > 0:
+            verification_score = (verified_claims + verified_units) / total_items
+        else:
+            verification_score = 1.0  # No claims to verify
+        
+        # Determine if response is verified
+        is_verified = verification_score >= self.similarity_threshold
+        
+        # Generate explanation or modified response if needed
+        if not is_verified:
+            # Flag unsupported claims and incorrect units
+            modified_response = self._highlight_engineering_issues(
+                model_response, 
+                [v["claim"] for v in verification_results if not v["verified"]],
+                [v["unit_value"] for v in unit_verification_results if not v["verified"]]
+            )
+        else:
+            modified_response = model_response
+            
+        verification_details = {
+            "verification_score": verification_score,
+            "claims_verified": verified_claims,
+            "total_claims": len(claims),
+            "units_verified": verified_units,
+            "total_units": len(units_and_values),
+            "claim_details": verification_results,
+            "unit_details": unit_verification_results
+        }
+        
+        return is_verified, modified_response, verification_details
+    
+    def _extract_technical_claims(self, text: str) -> List[str]:
+        """Extract technical claims from civil engineering text."""
+        sentences = re.split(r'(?<=[.!?])\s+', text)
+        
+        # Filter for sentences that likely contain technical claims
+        claims = []
+        for sentence in sentences:
+            # Look for sentences with technical terms or specifications
+            if re.search(r'\b(strength|load|capacity|stress|strain|factor|ratio|coefficient|standard|code|regulation|design|analysis|foundation|structure|concrete|steel|timber|masonry|soil|water|pressure|force|moment|deflection|displacement|settlement|safety|stability)\b', sentence.lower()):
+                if len(sentence.split()) > 5:  # Ignore very short sentences
+                    claims.append(sentence)
+        
+        return claims
+    
+    def _extract_units_and_values(self, text: str) -> List[str]:
+        """Extract units and numerical values from text."""
+        unit_values = []
+        
+        # Flatten the units list
+        all_units = []
+        for unit_list in self.engineering_units.values():
+            all_units.extend(unit_list)
+        
+        # Look for numbers followed by units
+        for unit in all_units:
+            # Escape special characters in unit for regex
+            escaped_unit = re.escape(unit)
+            # Find patterns like "123 mm" or "123.45 kN/m²"
+            matches = re.finditer(r'\b(\d+(?:\.\d+)?)\s*' + escaped_unit + r'\b', text)
+            for match in matches:
+                unit_values.append(match.group(0))
+        
+        return unit_values
+    
+    def _verify_technical_claim(self, claim: str, context: str) -> bool:
+        """
+        Verify if a technical claim is supported by engineering context.
+        """
+        # Extract key technical terms from the claim
+        technical_terms = re.findall(r'\b(strength|load|capacity|stress|strain|factor|ratio|coefficient|standard|code|regulation|design|analysis|foundation|structure|concrete|steel|timber|masonry|soil|water|pressure|force|moment|deflection|displacement|settlement|safety|stability)\b', claim.lower())
+        
+        # Extract numbers from the claim
+        numbers = re.findall(r'\b\d+(?:\.\d+)?\b', claim)
+        
+        # If claim has no technical terms or numbers, consider it non-technical
+        if not technical_terms and not numbers:
+            return True
+            
+        # Check for term presence in context
+        term_matches = 0
+        for term in technical_terms:
+            if re.search(r'\b' + re.escape(term) + r'\b', context.lower()):
+                term_matches += 1
+                
+        # Check for number presence in context
+        number_matches = 0
+        for number in numbers:
+            if re.search(r'\b' + re.escape(number) + r'\b', context):
+                number_matches += 1
+        
+        # Calculate match scores
+        if technical_terms:
+            term_score = term_matches / len(technical_terms)
+        else:
+            term_score = 1.0
+            
+        if numbers:
+            number_score = number_matches / len(numbers)
+        else:
+            number_score = 1.0
+            
+        # Weighted average of scores (terms more important than exact numbers)
+        final_score = (term_score * 0.7) + (number_score * 0.3)
+        
+        return final_score >= 0.6
+    
+    def _verify_unit_and_value(self, unit_value: str, context: str) -> bool:
+        """
+        Verify if a unit and its value are consistent with the context.
+        """
+        # Check if the exact unit-value pair appears in context
+        if re.search(r'\b' + re.escape(unit_value) + r'\b', context):
+            return True
+            
+        # Extract the number and unit
+        match = re.match(r'(\d+(?:\.\d+)?)\s*(\w+(?:/\w+)?)', unit_value)
+        if not match:
+            return False
+            
+        value, unit = match.groups()
+        
+        # Look for the same unit with similar values in context
+        # Find all numbers with this unit in the context
+        context_values = re.findall(r'(\d+(?:\.\d+)?)\s*' + re.escape(unit) + r'\b', context)
+        
+        if not context_values:
+            # Unit not found in context
+            return False
+            
+        # Convert to float for comparison
+        try:
+            float_value = float(value)
+            # Check if any value in context is close (within 10%)
+            for context_value in context_values:
+                try:
+                    context_float = float(context_value)
+                    # Allow 10% deviation
+                    if abs(float_value - context_float) / max(float_value, context_float) <= 0.1:
+                        return True
+                except ValueError:
+                    continue
+        except ValueError:
+            return False
+            
+        return False
+    
+    def _highlight_engineering_issues(self, response: str, unsupported_claims: List[str], incorrect_units: List[str]) -> str:
+        """Mark engineering issues in the response."""
+        modified = response
+        
+        # Mark unsupported technical claims
+        for claim in unsupported_claims:
+            if claim in modified:
+                modified = modified.replace(
+                    claim, 
+                    f"[UNVERIFIED CLAIM: {claim}]"
+                )
+        
+        # Mark potentially incorrect units and values
+        for unit_value in incorrect_units:
+            if unit_value in modified:
+                modified = modified.replace(
+                    unit_value, 
+                    f"[UNVERIFIED VALUE: {unit_value}]"
+                )
+        
+        # Add verification note
+        if unsupported_claims or incorrect_units:
+            modified += "\n\n[Note: Some technical claims or values in this response could not be verified against engineering documents. Please verify critical values with appropriate standards and codes.]"
+            
+        return modified
+
+# Usage example
+"""
+verifier = CivilEngineeringRAGVerifier()
+verified, modified_response, details = verifier.verify_response(
+    model_response="The concrete mix requires a water-cement ratio of 0.45 and a minimum compressive strength of 30 MPa at 28 days.",
+    retrieved_documents=[{"content": "Standard concrete mix design specifies a w/c ratio between 0.40 and 0.50, with expected 28-day strength of 25-35 MPa."}]
+)
+"""

config.json

@@ -1 +1 @@
-{"model_type": "llama", "architecture": "LlamaForCausalLM", "version": "1.0"}
+{"model_type": "pythia", "architecture": "GPTNeoXForCausalLM", "version": "1.0"}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7f1fcb11bdd243a23d7cc1666727df7eac5e1c8b994cff6423e35ccb62e2f307
-size 32
+oid sha256:cf538c1c59acecbf1d92197341c5d2e8ff20cbd0bfafd193f22cd94015ebbbed
+size 60

tokenizer.json

@@ -1 +1 @@
-{"model": "tinyllama", "type": "bpe"}
+{"model": "pythia", "type": "bpe"}