scraped_kb_articles/auto-loader-does-not-pick-up-files-for-processing-when-uploading-via-an-azure-function.json

{
    "url": "https://kb.databricks.com/en_US/delta-live-tables/auto-loader-does-not-pick-up-files-for-processing-when-uploading-via-an-azure-function",
    "title": "Título do Artigo Desconhecido",
    "content": "Problem\nWhen you upload files to a source location using an Azure function in Auto Loader, Auto Loader does not pick up the files for processing. The files are also not available in the queue, which Auto Loader sets automatically. The process does work, however, with manual intervention.\nCause\nAuto Loader listens for the ‘\nFlushWithClose\n’ event to process a file in file notification mode. The Azure function used for uploading files to a source location does not set the\nClose\nproperty in File Flush options, so EventBridge does not send the necessary event.\nSolution\nModify your Azure function to set the File Flush option with the\n'\nCLOSE\n' parameter.\nVerify that the\nFlushWithClose\nevent is generated in the Azure Queue after uploading the files to the source location.\nUse Directory listing mode until the file notification issue is resolved, then switch to file notification.\nMonitor the diagnostic logging on the storage Blob and Queue to identify any issues with the file upload process.\nFor more information, please review the\nWhat is Auto Loader file notification mode?\nand\nDataLakeFileFlushOptions.Close Property\ndocumentation."
}