Visual Similarity in Adult Word Extension and Artificial Word Learning
Abstract
Visual similarity measured by a self-supervised visual encoder partially explains word generalization in adults and artificial learners, while the composition of teaching examples contributes additional structure beyond that distance.
When two objects share a name, how much does their visual similarity explain, and what do the teaching examples add? First, we reanalyzed trial-level data from a large adult word-extension study (Lewis & Frank, 2018), computing each test image's distance to the examples shown in the space of a frozen self-supervised visual encoder (DINOv2). Distance was associated with extension (log odds -3.23 per standard deviation; 520 people), and an exploratory post hoc analysis suggested that this association persisted within the taught domain after controlling for taxonomic level and teaching condition. Example composition also mattered: after three examples, adults extended a word to 1.1% of superordinate items when the examples were subordinate and to 77.5% when they were superordinate. Second, in an artificial learner using the same encoder, words joining two categories were learned less accurately the farther apart the categories were (84.7%, 65.4%, and 53.0% naming accuracy for near, mid, and far pairs). The encoder's distances correlated with human similarity and confusability measures from three datasets (rho = .501-.795), but this representational alignment did not translate into conclusive evidence that human-derived distances predict learner difficulty, and a second encoder trained on one child's experience reproduced the tier ordering but not the pair-level effect. The secondary human data were analyzed under an internal plan fixed before the confirmatory analyses (not externally preregistered). Visual similarity accounts for part of how generalization is organized in both systems, while the composition of examples contributes what the distance we used does not capture.
Community
When two objects share a name, how much does their visual similarity explain, and what do the teaching examples add? We reanalyze trial-level data from a large adult word-extension study using distances in a frozen DINOv2 encoder, and test an artificial word learner built on the same encoder. Datasets: https://huggingface.co/collections/MrPatoVera/lexical-consensus-6af31e7f9870b8e83e668c49
Models citing this paper 0
No model linking this paper
Datasets citing this paper 2
MrPatoVera/lexical-consensus-data-cifar10h
Spaces citing this paper 0
No Space linking this paper