app/scripts/latex-importer/README.md

# LaTeX to MDX Toolkit

Complete LaTeX to MDX (Markdown + JSX) conversion optimized for Astro with advanced support for references, interactive equations, and components.

## 🚀 Quick Start

```bash
# Complete LaTeX → MDX conversion with all features
node index.mjs

# For step-by-step debugging
node latex-converter.mjs    # LaTeX → Markdown
node mdx-converter.mjs      # Markdown → MDX
```

## 📁 Structure

```
latex-to-mdx/
├── index.mjs                    # Complete LaTeX → MDX pipeline
├── latex-converter.mjs          # LaTeX → Markdown with Pandoc
├── mdx-converter.mjs           # Markdown → MDX with Astro components
├── reference-preprocessor.mjs  # LaTeX references cleanup
├── post-processor.mjs          # Markdown post-processing
├── bib-cleaner.mjs            # Bibliography cleaner
├── filters/
│   └── equation-ids.lua        # Pandoc filter for KaTeX equations
├── input/                      # LaTeX sources
│   ├── main.tex
│   ├── main.bib
│   └── sections/
└── output/                     # Results
    ├── main.md                 # Intermediate Markdown
    └── main.mdx               # Final MDX for Astro
```

## ✨ Key Features

### 🎯 **Smart References**
- **Invisible anchors**: Automatic conversion of `\label{}` to `<span id="..." style="position: absolute;"></span>`
- **Clean links**: Identifier cleanup (`:` → `-`, removing prefixes `sec:`, `fig:`, `eq:`)
- **Cross-references**: Full support for `\ref{}` with functional links

### 🧮 **Interactive Equations**
- **KaTeX IDs**: Conversion of `\label{eq:...}` to `\htmlId{id}{equation}` 
- **Equation references**: Clickable links to mathematical equations
- **Advanced KaTeX support**: `trust: true` configuration for `\htmlId{}`

### 🎨 **Automatic Styling**  
- **Highlights**: `\highlight{text}` → `<span class="highlight">text</span>`
- **Auto cleanup**: Removal of numbering `(1)`, `(2)`, etc.
- **Astro components**: Images → `ResponsiveImage` with automatic imports

### 🔧 **Robust Pipeline**
- **LaTeX preprocessor**: Reference cleanup before Pandoc
- **Lua filter**: Equation processing in Pandoc AST  
- **Post-processor**: Markdown cleanup and optimization
- **MDX converter**: Final transformation with Astro components

## 📊 Example Workflow

```bash
# 1. Prepare LaTeX sources
cp my-paper/* input/

# 2. Complete automatic conversion
node index.mjs

# 3. Generated results
ls output/
# → main.md (Intermediate Markdown)  
# → main.mdx (Final MDX for Astro)
# → assets/image/ (extracted images)
```

### 📋 Conversion Result

The pipeline generates an MDX file optimized for Astro with:

```mdx
---
title: "Your Article Title"
description: "Generated from LaTeX"
---

import ResponsiveImage from '../components/ResponsiveImage.astro';
import figure1 from '../assets/image/figure1.png';

## Section with invisible anchor
<span id="introduction" style="position: absolute;"></span>

Here is some text with <span class="highlight">highlighted words</span>.

Reference to an interactive [equation](#equation-name).

Equation with KaTeX ID:
$$\htmlId{equation-name}{E = mc^2}$$

<ResponsiveImage src={figure1} alt="Description" />
```

## ⚙️ Required Astro Configuration

To use equations with IDs, add to `astro.config.mjs`:

```javascript
import rehypeKatex from 'rehype-katex';

export default defineConfig({
  markdown: {
    rehypePlugins: [
      [rehypeKatex, { trust: true }], // ← Important for \htmlId{}
    ],
  },
});
```

## 🛠️ Prerequisites

- **Node.js** with ESM support
- **Pandoc** (`brew install pandoc`)
- **Astro** to use the generated MDX

## 🎯 Technical Architecture

### 4-Stage Pipeline

1. **LaTeX Preprocessing** (`reference-preprocessor.mjs`)
   - Cleanup of `\label{}` and `\ref{}`
   - Conversion `\highlight{}` → CSS spans
   - Removal of prefixes and problematic characters

2. **Pandoc + Lua Filter** (`equation-ids.lua`)
   - LaTeX → Markdown conversion with `gfm+tex_math_dollars+raw_html`
   - Equation processing: `\label{eq:name}` → `\htmlId{name}{equation}`
   - Automatic image extraction

3. **Markdown Post-processing** (`post-processor.mjs`)
   - KaTeX, Unicode, grouping commands cleanup
   - Attribute correction with `:` 
   - Code snippet injection

4. **MDX Conversion** (`mdx-converter.mjs`)
   - Images transformation → `ResponsiveImage`
   - HTML span escaping correction
   - Automatic imports generation
   - MDX frontmatter

## 📊 Conversion Statistics

For a typical scientific document:
- **87 labels** detected and processed
- **48 invisible anchors** created  
- **13 highlight spans** with CSS class
- **4 equations** with `\htmlId{}` KaTeX
- **40 images** converted to components

## ✅ Project Status

### 🎉 **Complete Features**
- ✅ **LaTeX → MDX Pipeline**: Full end-to-end functional conversion
- ✅ **Cross-document references**: Perfectly functional internal links  
- ✅ **Interactive equations**: KaTeX support with clickable IDs
- ✅ **Automatic styling**: Highlights and Astro components
- ✅ **Robustness**: Automatic cleanup of all escaping
- ✅ **Optimization**: Clean code without unnecessary elements

### 🚀 **Production Ready**
The toolkit is now **100% operational** for converting complex scientific LaTeX documents to MDX/Astro with all advanced features (references, interactive equations, styling).