openbmb

/

MiniCPM4-8B

like 283

Follow

OpenBMB 2.97k

Description

Fixes Bug: Inference fails on older transformers versions due to DynamicLayer import and incorrect past_key_values initialization · Issue #339

This PR addresses two compatibility bugs that prevent MiniCPM4 from running on older transformers versions (specifically versions prior to 4.54.1).

Changes Made:

1. Added CacheLayerMixin and DynamicLayer
   • Directly integrated these base classes into modeling_minicpm.py. This ensures that environments lacking these specific cache utilities in transformers.cache_utils will still function seamlessly, avoiding the ImportError.
   • commit1
2. Fixed past_key_values evaluation in MiniCPMModel.forward
   • Modified the cache validation logic to explicitly accept None during the initial forward pass.
   • Updated the cache initialization to properly instantiate InfLLMv2Cache or DynamicCache when past_key_values is None.
   • Removed the unused use_legacy_cache variable at the end of the forward method to clean up the return logic.
   • commit2

📋 Code Snippet of Core Logic Change:

# Before
if use_cache:
    use_legacy_cache = not isinstance(past_key_values, Cache)
    if use_legacy_cache:
        raise ValueError(...) 

# After
if use_cache:
    # Reject old tuple-style cache, but allow None (first forward pass)
    if past_key_values is not None and not isinstance(past_key_values, Cache):
        raise ValueError(
            'You must use the new past_key_values format, such as the Cache class, instead of the old tuple format.'
        )

    # Initialize cache if None (first forward pass)
    if past_key_values is None:
        if getattr(self.config, "sparse_config", None) is not None and torch.cuda.is_available():
            past_key_values = InfLLMv2Cache(config=self.config, num_hidden_layers=self.config.num_hidden_layers)
        else:
            past_key_values = DynamicCache()