Enhance Dockerfile and Streamlit app for Hugging Face directory management

- Updated Dockerfile to create additional directories for Hugging Face caching with appropriate permissions.
- Enhanced Streamlit app to configure environment variables for Hugging Face and implement robust directory creation with error handling and logging.
- Improved fallback mechanisms for directory creation to ensure stability in various environments.

Dockerfile

@@ -10,13 +10,17 @@ RUN apt-get update && apt-get install -y \
     && rm -rf /var/lib/apt/lists/*
 
 # Create necessary directories with proper permissions
-RUN mkdir -p /app/.streamlit /tmp/docling_temp /tmp/easyocr_models /tmp/cache /tmp/config /tmp/data && \
+RUN mkdir -p /app/.streamlit /tmp/docling_temp /tmp/easyocr_models /tmp/cache /tmp/config /tmp/data /tmp/huggingface /tmp/huggingface_cache /tmp/transformers_cache /tmp/datasets_cache && \
     chmod 755 /app/.streamlit && \
     chmod 777 /tmp/docling_temp && \
     chmod 777 /tmp/easyocr_models && \
     chmod 777 /tmp/cache && \
     chmod 777 /tmp/config && \
-    chmod 777 /tmp/data
+    chmod 777 /tmp/data && \
+    chmod 777 /tmp/huggingface && \
+    chmod 777 /tmp/huggingface_cache && \
+    chmod 777 /tmp/transformers_cache && \
+    chmod 777 /tmp/datasets_cache
 
 COPY requirements.txt ./
 COPY src/ ./src/

TROUBLESHOOTING.md

@@ -95,4 +95,34 @@ The app automatically sets these environment variables:
 - `HOME=/tmp/docling_temp` (or fallback)
 - `XDG_CACHE_HOME=/tmp/cache` (or fallback)
 - `XDG_CONFIG_HOME=/tmp/config` (or fallback)
-- `XDG_DATA_HOME=/tmp/data` (or fallback) 
+- `XDG_DATA_HOME=/tmp/data` (or fallback) 
+
+### Hugging Face Hub Permission Errors
+
+If you encounter Hugging Face Hub permission errors like:
+```
+PermissionError: [Errno 13] Permission denied: '/.cache'
+```
+
+The app now handles these by:
+1. Setting `HF_HOME`, `HF_CACHE_HOME`, `TRANSFORMERS_CACHE`, and `HF_DATASETS_CACHE` to writable directories
+2. Creating all necessary Hugging Face cache directories with proper permissions
+3. Using fallback directories if the primary ones fail
+
+### Environment Variables
+
+The app automatically sets these environment variables:
+- `STREAMLIT_SERVER_FILE_WATCHER_TYPE=none`
+- `STREAMLIT_SERVER_HEADLESS=true`
+- `STREAMLIT_BROWSER_GATHER_USAGE_STATS=false`
+- `STREAMLIT_SERVER_ENABLE_CORS=false`
+- `STREAMLIT_SERVER_ENABLE_XSRF_PROTECTION=false`
+- `EASYOCR_MODULE_PATH=/tmp/easyocr_models` (or fallback)
+- `HOME=/tmp/docling_temp` (or fallback)
+- `XDG_CACHE_HOME=/tmp/cache` (or fallback)
+- `XDG_CONFIG_HOME=/tmp/config` (or fallback)
+- `XDG_DATA_HOME=/tmp/data` (or fallback)
+- `HF_HOME=/tmp/huggingface` (or fallback)
+- `HF_CACHE_HOME=/tmp/huggingface_cache` (or fallback)
+- `TRANSFORMERS_CACHE=/tmp/transformers_cache` (or fallback)
+- `HF_DATASETS_CACHE=/tmp/datasets_cache` (or fallback) 

src/streamlit_app.py

@@ -97,6 +97,29 @@ for env_var in ['XDG_CACHE_HOME', 'XDG_CONFIG_HOME', 'XDG_DATA_HOME']:
     except Exception as e:
         logging.warning(f"Could not create directory for {env_var}: {e}")
 
+# Configure Hugging Face Hub to use writable directories
+os.environ['HF_HOME'] = os.path.join(TEMP_DIR, 'huggingface')
+os.environ['HF_CACHE_HOME'] = os.path.join(TEMP_DIR, 'huggingface_cache')
+os.environ['TRANSFORMERS_CACHE'] = os.path.join(TEMP_DIR, 'transformers_cache')
+os.environ['HF_DATASETS_CACHE'] = os.path.join(TEMP_DIR, 'datasets_cache')
+
+# Create Hugging Face directories
+hf_dirs = ['HF_HOME', 'HF_CACHE_HOME', 'TRANSFORMERS_CACHE', 'HF_DATASETS_CACHE']
+for env_var in hf_dirs:
+    try:
+        os.makedirs(os.environ[env_var], exist_ok=True)
+        logging.info(f"Created Hugging Face directory for {env_var}: {os.environ[env_var]}")
+    except Exception as e:
+        logging.warning(f"Could not create Hugging Face directory for {env_var}: {e}")
+        # Fallback to /tmp if the temp directory fails
+        fallback_path = os.path.join('/tmp', env_var.lower())
+        os.environ[env_var] = fallback_path
+        try:
+            os.makedirs(fallback_path, exist_ok=True)
+            logging.info(f"Using fallback Hugging Face directory for {env_var}: {fallback_path}")
+        except Exception as e2:
+            logging.error(f"Failed to create fallback Hugging Face directory for {env_var}: {e2}")
+
 # Log startup information
 logging.info("=" * 50)
 logging.info("Docling Streamlit App Starting")