why in-context learning excels in tabular data
#9
by BILL-SUN318 - opened
I'm not an expert but I know a little about ML/DL/LLM. As is standard practice for LLMs, a SOTA model is produced via large-scale pretraining followed by fine-tuning. However, as observed in many studies on SOTA tabular-data models, the training data is fed into the pretrained model as input (or context).
What accounts for this difference? Is it the design of the training loss objective that makes the difference?