checkpoint first presentable hallucination demo

README.md

@@ -34,39 +34,74 @@ This tool uses **generative inference** with adversarially robust neural network
 
 ## Usage
 
-1. **Select an example illusion** or upload your own image
-2. **Click "Load Parameters"** to set optimal prediction settings
-3. **Click "Run Generative Inference"** to predict the hallucination
-4. **View the results**: The model will show what perceptual effects it predicts humans will experience
+1. **Choose an input**: Pick a pre-configured example illusion from the dropdown, or upload your own image.
+2. **Load Parameters**: Click **"Load Parameters"** to fill in optimal prediction settings for that example (or adjust them manually).
+3. **Select the affected part of the visual field**: Set where the model should focus by clicking on the input image or the mask preview to define the mask center, then adjust **Mask center X/Y**, **Mask radius**, and **Mask sigma** in the Adaptive Gaussian mask section if needed. The preview circle shows the region that will receive stronger constraint during inference.
+4. **Run inference**: Click **"Run Generative Inference"** to start the prediction. Progress and intermediate steps are shown in the interface.
+5. **View results**: Inspect the predicted perceptual effects, visualizations, and any generated outputs in the result panels.
 
 ## Scientific Background
 
 This demo is based on research showing that adversarially robust neural networks develop perceptual representations similar to human vision. By using generative inference (optimizing images to maximize model confidence), we can reveal what perceptual structures the network expects to see—which often matches what humans hallucinate or perceive in ambiguous images.
 
+## Prerequisites
+
+- **Python** 3.8 or higher
+- **pip** (Python package manager)
+
 ## Installation
 
-To run this demo locally:
+1. **Clone the repository**
 
-```bash
-# Clone the repository
-git clone https://huggingface.co/spaces/ttoosi/Human_Hallucination_Prediction
-cd Human_Hallucination_Prediction
+   ```bash
+   git clone https://huggingface.co/spaces/ttoosi/Human_Hallucination_Prediction
+   cd Human_Hallucination_Prediction
+   ```
+
+2. **Create a virtual environment** (recommended)
+
+   ```bash
+   python -m venv venv
+   source venv/bin/activate   # On Windows: venv\Scripts\activate
+   ```
+
+3. **Install dependencies**
+
+   ```bash
+   pip install -r requirements.txt
+   ```
 
-# Install dependencies
-pip install -r requirements.txt
+4. **Run the app**
 
-# Run the app
-python app.py
+   ```bash
+   python app.py
+   ```
+
+   Optional: specify a port with `--port` (default is 7860):
+
+   ```bash
+   python app.py --port 8861
+   ```
+
+The web app will be available at **http://localhost:7860** (or the port you specified).
+
+**Note:** Model weights (e.g. robust ResNet50) are downloaded automatically from Hugging Face on first run and cached in the `models/` directory. The app also creates a `stimuli/` directory for example images.
+
+### Running with Docker
+
+```bash
+docker build -t human-hallucination-prediction .
+docker run -p 7860:7860 human-hallucination-prediction
 ```
 
-The web app will be available at http://localhost:7860.
+Then open http://localhost:7860 in your browser.
 
 ## The Prediction Process
 
-1. **Input**: Start with an ambiguous or illusion-inducing image
-2. **Generative Inference**: The robust neural network iteratively modifies the image to maximize its confidence
-3. **Prediction**: The modifications reveal what perceptual structures the network expects—predicting what humans will hallucinate
-4. **Visualization**: View the predicted hallucination emerging step-by-step
+1. **Input**: You provide an ambiguous or illusion-inducing image (or use a built-in example).
+2. **Generative inference**: The adversarially robust network iteratively updates the image to maximize its confidence, guided by your chosen parameters (model, layer, noise, step size, etc.).
+3. **Prediction**: The resulting changes reveal the perceptual structures the network expects—which correspond to what humans tend to hallucinate or perceive in such images.
+4. **Visualization**: The interface shows the predicted hallucination and intermediate steps as the optimization runs.
 
 ## Models
 

app.py

@@ -33,6 +33,66 @@ model = GenerativeInferenceModel()
 
 # Define example images and their parameters with updated values from the research
 examples = [
+    {
+        "image": os.path.join("stimuli", "farm1.jpg"),
+        "name": "farm1",
+        "wiki": "https://en.wikipedia.org/wiki/Visual_perception",
+        "papers": [
+            "[Adversarially Robust Vision](https://github.com/MadryLab/robustness)",
+            "[Generative Inference](https://doi.org/10.1016/j.tics.2003.08.003)"
+        ],
+        "method": "Prior-Guided Drift Diffusion",
+        "reverse_diff": {
+            "model": "resnet50_robust",
+            "layer": "all",
+            "initial_noise": 0.0,
+            "diffusion_noise": 0.02,
+            "step_size": 1.0,
+            "iterations": 501,
+            "epsilon": 40.0
+        },
+        "inference_normalization": "off",
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 0.3,
+        "eps_max_mult": 300.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 10.0,
+        "step_min_mult": 1.0,
+    },
+    {
+        "image": os.path.join("stimuli", "ArtGallery1.jpg"),
+        "name": "ArtGallery1",
+        "wiki": "https://en.wikipedia.org/wiki/Visual_perception",
+        "papers": [
+            "[Adversarially Robust Vision](https://github.com/MadryLab/robustness)",
+            "[Generative Inference](https://doi.org/10.1016/j.tics.2003.08.003)"
+        ],
+        "method": "Prior-Guided Drift Diffusion",
+        "reverse_diff": {
+            "model": "resnet50_robust",
+            "layer": "layer4",
+            "initial_noise": 0.5,
+            "diffusion_noise": 0.002,
+            "step_size": 0.1,
+            "iterations": 501,
+            "epsilon": 40.0
+        },
+        "inference_normalization": "off",
+        "use_adaptive_eps": False,
+        "use_adaptive_step": True,
+        "mask_center_x": 0.0,
+        "mask_center_y": -1.0,
+        "mask_radius": 0.1,
+        "mask_sigma": 0.2,
+        "eps_max_mult": 30.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 100.0,
+        "step_min_mult": 1.0,
+    },
     {
         "image": os.path.join("stimuli", "urbanoffice1.jpg"),
         "name": "UrbanOffice1",
@@ -80,7 +140,17 @@ examples = [
             "step_size": 1.0,
             "iterations": 101,
             "epsilon": 20.0
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "Kanizsa_square.jpg"),
@@ -99,7 +169,17 @@ examples = [
             "step_size": 0.64,
             "iterations": 100,
             "epsilon": 5.0
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "CornsweetBlock.png"),
@@ -119,7 +199,17 @@ examples = [
             "step_size": 0.8,
             "iterations": 51,
             "epsilon": 20.0
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "face_vase.png"),
@@ -138,7 +228,17 @@ examples = [
             "step_size": 0.58,
             "iterations": 100,
             "epsilon": 0.81
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "Confetti_illusion.png"),
@@ -157,7 +257,17 @@ examples = [
             "step_size": 0.5,
             "iterations": 101,
             "epsilon": 20.0
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "EhresteinSingleColor.png"),
@@ -176,7 +286,17 @@ examples = [
             "step_size": 0.8,
             "iterations": 101,
             "epsilon": 20.0
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "GroupingByContinuity.png"),
@@ -195,7 +315,17 @@ examples = [
             "step_size": 0.4,
             "iterations": 101,
             "epsilon": 4.0
-        }
+        },
+        "use_adaptive_eps": False,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
+        "mask_center_y": 0.0,
+        "mask_radius": 0.2,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
+        "eps_min_mult": 1.0,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     },
     {
         "image": os.path.join("stimuli", "figure_ground.png"),
@@ -214,37 +344,17 @@ examples = [
             "step_size": 0.5,
             "iterations": 101,
             "epsilon": 3.0
-        }
-    },
-    {
-        "image": os.path.join("stimuli", "urbanoffice1.jpg"),
-        "name": "UrbanOffice1",
-        "wiki": "https://en.wikipedia.org/wiki/Visual_perception",
-        "papers": [
-            "[Adversarially Robust Vision](https://github.com/MadryLab/robustness)",
-            "[Generative Inference](https://doi.org/10.1016/j.tics.2003.08.003)"
-        ],
-        "method": "Prior-Guided Drift Diffusion",
-        "reverse_diff": {
-            "model": "resnet50_robust",
-            "layer": "all",
-            "initial_noise": 1.0,
-            "diffusion_noise": 0.002,
-            "step_size": 1.0,
-            "iterations": 500,
-            "epsilon": 40.0
         },
-        "inference_normalization": "off",
         "use_adaptive_eps": False,
-        "use_adaptive_step": True,
-        "mask_center_x": 0.5,
+        "use_adaptive_step": False,
+        "mask_center_x": 0.0,
         "mask_center_y": 0.0,
         "mask_radius": 0.2,
-        "mask_sigma": 0.2,
-        "eps_max_mult": 20.0,
+        "mask_sigma": 1.0,
+        "eps_max_mult": 1.0,
         "eps_min_mult": 1.0,
-        "step_max_mult": 50.0,
-        "step_min_mult": 0.2,
+        "step_max_mult": 1.0,
+        "step_min_mult": 1.0,
     }
 ]
 
@@ -398,6 +508,10 @@ def draw_mask_overlay(image, center_x, center_y, radius):
 # Helper function to apply example parameters (adaptive mask off by default unless example defines it)
 def apply_example(example):
     rd = example["reverse_diff"]
+    mcx = example.get("mask_center_x", 0.0)
+    mcy = example.get("mask_center_y", 0.0)
+    mrad = example.get("mask_radius", 0.3)
+    mask_img = draw_mask_overlay(example["image"], mcx, mcy, mrad)
     return [
         example["image"],
         rd.get("model", "resnet50_robust"),
@@ -410,14 +524,15 @@ def apply_example(example):
         rd["layer"],
         example.get("use_adaptive_eps", False),
         example.get("use_adaptive_step", False),
-        example.get("mask_center_x", 0.0),
-        example.get("mask_center_y", 0.0),
+        mcx,
+        mcy,
         example.get("mask_radius", 0.3),
         example.get("mask_sigma", 0.2),
         example.get("eps_max_mult", 4.0),
         example.get("eps_min_mult", 1.0),
         example.get("step_max_mult", 4.0),
         example.get("step_min_mult", 1.0),
+        mask_img,
         gr.Group(visible=True),
     ]
 
@@ -433,17 +548,15 @@ with gr.Blocks(title="Human Hallucination Prediction", css="""
 }
 """) as demo:
     gr.Markdown("# Human Hallucination Prediction")
-    gr.Markdown("**Predict what visual hallucinations humans will experience** using adversarially robust neural networks. This demo forecasts perceptual phenomena like illusory contours, figure-ground reversals, and Gestalt effects before humans report them.")
+    gr.Markdown("**Predict what visual hallucinations humans may experience** using neural networks.")
     
     gr.Markdown("""
     **How to predict hallucinations:**
-    1. **Select an example illusion** below and click "Load Parameters" to set optimal prediction settings
-    2. **Click "Run Generative Inference"** to predict what hallucination humans will perceive
+    1. **Select an example image** below and click "Load Parameters" to set the prediction settings
+    2. **Click "Run Generative Inference"** to predict what hallucination humans may perceive
     3. **View the prediction**: Watch as the model reveals the perceptual structures it expects—matching what humans typically hallucinate
-    4. **Upload your own images** to test if they will induce hallucinations in human observers
+    4. **You can upload your own images**
     """)
-    
-    # Main processing interface
     with gr.Row():
         with gr.Column(scale=1):
             # Inputs
@@ -503,7 +616,7 @@ with gr.Blocks(title="Human Hallucination Prediction", css="""
                     mask_center_y_slider = gr.Slider(minimum=-1.0, maximum=1.0, value=0.0, step=0.05, label="Mask center Y")
                 with gr.Row():
                     mask_radius_slider = gr.Slider(minimum=0.01, maximum=1.0, value=0.2, step=0.01, label="Mask radius (flat region size)")
-                    mask_sigma_slider = gr.Slider(minimum=0.05, maximum=0.5, value=0.2, step=0.01, label="Mask sigma (fall-off outside radius)")
+                    mask_sigma_slider = gr.Slider(minimum=0.05, maximum=1.0, value=0.2, step=0.01, label="Mask sigma (fall-off outside radius)")
                 with gr.Row():
                     eps_max_mult_slider = gr.Slider(minimum=0.1, maximum=350.0, value=20.0, step=0.1, label="Epsilon: multiplier at center")
                     eps_min_mult_slider = gr.Slider(minimum=0.1, maximum=10.0, value=1.0, step=0.1, label="Epsilon: multiplier at periphery")
@@ -542,6 +655,7 @@ with gr.Blocks(title="Human Hallucination Prediction", css="""
                         mask_radius_slider, mask_sigma_slider,
                         eps_max_mult_slider, eps_min_mult_slider,
                         step_max_mult_slider, step_min_mult_slider,
+                        mask_preview,
                         params_section,
                     ],
                 )