tiiuae

/

falcon-7b

like 1.1k

Follow

Technology Innovation Institute 2.08k

I have trained falcon-7b and pushed the finetuned model to hub, now when i am trying to use Langchain with my new model- which is saved at "tdicommons/falcon_7b_6_27_2023".
I am using following code to proceed with my objective:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, AutoModelForSeq2SeqLM

model_id = "tdicommons/falcon_7b_6_27_2023" # our finetuned model from HF here.
tokenizer = AutoTokenizer.from_pretrained(model_id,trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id, load_in_8bit=True)

pipe = pipeline(
    "text-generation",
    model=model, 
    tokenizer=tokenizer, 
    max_length=100
)

local_llm = HuggingFacePipeline(pipeline=pipe)```

I am getting an error, which is something i can't understand - can anyone help me on this ?? 

Error: ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-19-1462851e614d> in <cell line: 8>()
      6 
      7 model_id = "tdicommons/falcon_7b_6_27_2023" # our finetuned model from HF here.
----> 8 tokenizer = AutoTokenizer.from_pretrained(model_id,trust_remote_code=True)
      9 model = AutoModelForSeq2SeqLM.from_pretrained(model_id, load_in_8bit=True)
     10 

/usr/local/lib/python3.10/dist-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    717                     )
    718 
--> 719         raise ValueError(
    720             f"Unrecognized configuration class {config.__class__} to build an AutoTokenizer.\n"
    721             f"Model type should be one of {', '.join(c.__name__ for c in TOKENIZER_MAPPING.keys())}."

ValueError: Unrecognized configuration class <class 'transformers_modules.tiiuae.falcon-7b.2f5c3cd4eace6be6c0f12981f377fb35e5bf6ee5.configuration_RW.RWConfig'> to build an AutoTokenizer.
Model type should be one of AlbertConfig, AlignConfig, BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BlipConfig, Blip2Config, BloomConfig, BridgeTowerConfig, CamembertConfig, CanineConfig, ChineseCLIPConfig, ClapConfig, CLIPConfig, CLIPSegConfig, CodeGenConfig, ConvBertConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, DPRConfig, ElectraConfig, ErnieConfig, ErnieMConfig, EsmConfig, FlaubertConfig, FNetConfig, FSMTConfig, FunnelConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, GPTSanJapaneseConfig, GroupViTConfig, HubertConfig, IBertConfig, JukeboxConfig, LayoutLMConfig, LayoutLMv2Config, LayoutLMv3Config, LEDConfig, LiltConfig, LlamaConfig, LongformerConfig, LongT5Config, LukeConfig, LxmertConfig, M2M100Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MgpstrConfig, MobileBertConfig, MPNetConfig, MT5Config, MvpConfig, NezhaConfig, NllbMoeConfig, NystromformerConfig, OneFormerConfig, OpenAIGPTConfig, OPTConfig, OwlViTConfig, PegasusConfig, PegasusXConfig, PerceiverConfig, Pix2StructConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, RagConfig, RealmConfig, ReformerConfig, RemBertConfig, RetriBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, Speech2TextConfig, Speech2Text2Config, SpeechT5Confi...