deepapaikar

/

LlamaKatz-Pretrain

like 0

• "torch._utils._rebuild_tensor_v2",
• "torch._tensor._rebuild_from_type_v2",
• "collections.OrderedDict",
• "torch.FloatStorage",
• "torch.Tensor"

How to fix it?

• "torch._utils._rebuild_parameter",
• "torch._utils._rebuild_tensor_v2",
• "torch.nn.modules.container.ModuleDict",
• "collections.OrderedDict",
• "lit_llama.model.Block",
• "bitsandbytes.autograd._functions.MatmulLtState",
• "torch.nn.modules.sparse.Embedding",
• "__builtin__.set",
• "torch.nn.modules.container.ModuleList",
• "torch.nn.modules.module._WrappedHook",
• "lit_llama.model.RMSNorm",
• "lit_llama.model.LLaMAConfig",
• "lit_llama.model.LLaMA",
• "lit_llama.model.MLP",
• "torch._utils._rebuild_parameter_with_state",
• "torch.FloatStorage",
• "lit_llama.model.CausalSelfAttention",
• "lit_llama.quantization.Linear8bitLt",
• "bitsandbytes.nn.modules.maybe_rearrange_weight",
• "torch.CharStorage",
• "torch.BFloat16Storage"

How to fix it?