RetentionLabs

/

TTT-Linear-350M-Base-Books-2k

like 0

Environment: transformers version(s) tested (5.13.0 and 5.12.1(Colab's default transformers version is (5.12.1)) fails)
Issue: The code to run the models in the colab notebook yields an attribute error when transformer 5.x version is used

# Codeline
from transformers import pipeline

pipe = pipeline("text-generation", model="RetentionLabs/TTT-Linear-350M-Base-Books-2k", trust_remote_code=True)

Traceback

 config.json: 100% 879/879 [00:00<00:00, 84.5kB/s]Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
WARNING:huggingface_hub.utils._http:Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
modeling_ttt.py: 100% 72.2k/72.2k [00:00<00:00, 5.89MB/s][transformers] A new version of the following files was downloaded from https://huggingface.co/RetentionLabs/TTT-Linear-350M-Base-Books-2k:
- modeling_ttt.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
model.safetensors: 100% 675M/675M [00:06<00:00, 135MB/s]---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_1285/2532382187.py in <cell line: 0>()
      2 from transformers import pipeline
      3 
----> 4 pipe = pipeline("text-generation", model="RetentionLabs/TTT-Linear-350M-Base-Books-2k", trust_remote_code=True)

6 frames/usr/local/lib/python3.12/dist-packages/transformers/pipelines/__init__.py in pipeline(task, model, config, tokenizer, feature_extractor, image_processor, video_processor, processor, revision, use_fast, token, device, device_map, dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs)
   1031     if isinstance(model, str):
   1032         model_classes = targeted_task["pt"]
-> 1033         model = load_model(
   1034             adapter_path if adapter_path is not None else model,
   1035             model_classes=model_classes,

/usr/local/lib/python3.12/dist-packages/transformers/pipelines/base.py in load_model(model, config, model_classes, task, **model_kwargs)
    231 
    232             try:
--> 233                 model = model_class.from_pretrained(model, **kwargs)
    234                 # Stop loading on the first successful load.
    235                 break

/usr/local/lib/python3.12/dist-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    389                 model_class.register_for_auto_class(auto_class=cls)
    390             model_class = add_generation_mixin_to_remote_model(model_class)
--> 391             return model_class.from_pretrained(
    392                 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    393             )

/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, weights_only, fusion_config, disable_mmap, *model_args, **kwargs)
   4281         config = copy.deepcopy(config)  # We do not want to modify the config inplace in from_pretrained.
   4282         with ContextManagers(model_init_context):
-> 4283             model = cls(config, *model_args, **model_kwargs)
   4284             patch_output_recorders(model)
   4285 

~/.cache/huggingface/modules/transformers_modules/RetentionLabs/TTT_hyphen_Linear_hyphen_350M_hyphen_Base_hyphen_Books_hyphen_2k/8c2af596ccf5f0d4a35fdd2bd89e2c21106d2acb/modeling_ttt.py in __init__(self, config)
   1517 
   1518         # Initialize weights and apply final processing
-> 1519         self.post_init()
   1520 
   1521     def get_input_embeddings(self):

/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py in post_init(self)
   1400             self._ep_plan = self.config.base_model_ep_plan.copy() if self.config.base_model_ep_plan is not None else {}
   1401         # Current submodel should register its tied weights
-> 1402         self.all_tied_weights_keys = self.get_expanded_tied_weights_keys(all_submodels=False)
   1403         # Current submodel should register its `_keep_in_fp32_modules`
   1404         self._keep_in_fp32_modules = set(self._keep_in_fp32_modules or [])

/usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py in get_expanded_tied_weights_keys(self, all_submodels)
   2622         # sure it does not contain a regex pattern, and finishing by "bias" or "weight" to make sure it's not a module)
   2623         common_case_regex = re.compile(r"^[A-Za-z0-9_\.]+(weight)|(bias)$")
-> 2624         if all(common_case_regex.match(k) for k in tied_mapping.keys() | tied_mapping.values()):
   2625             return tied_mapping.copy()
   2626 
AttributeError: 'list' object has no attribute 'keys'

Workaround: Downgrading transformers<5 fixes the issues. I tested it to work on v4.57.6
Root cause: transformers v5.x's post_init() now expects the tied-weights attribute to be a dict rather than a list.

Suggested Fix: Convert _tied_weights_keys from a list to a dict, mapping each weight name to the weight it's tied to.
Reference : https://github.com/huggingface/transformers/issues/43646