scraped_kb_articles/auto-loader-streaming-query-failure-with-unknownfieldexception-error.json

{
    "url": "https://kb.databricks.com/en_US/streaming/auto-loader-streaming-query-failure-with-unknownfieldexception-error",
    "title": "Título do Artigo Desconhecido",
    "content": "Problem\nYour Auto Loader streaming job fails with an\nUnknownFieldException\nerror when a new column is added to the source file of the stream.\nException: org.apache.spark.sql.catalyst.util.UnknownFieldException: Encountered unknown field(s) during parsing: <column name>\nCause\nAn\nUnknownFieldException\nerror occurs when Auto Loader detects the addition of new columns as it processes incoming data.\nThe addition of a new column causes the stream to stop and generates an\nUnknownFieldException\nerror.\nSolution\nSet your Auto Loader stream to use schema evolution to avoid this issue.\nFor more information, review the\nHow does Auto Loader schema evolution work?\n(\nAWS\n|\nAzure\n|\nGCP\n) documentation."
}