StarCoder in AWS Sagemaker? Error "gpt_bigcode" for not supporting Transformers last version

#41

by apol - opened May 25, 2023

BigCode org May 25, 2023

Hi, trying to deploy StarChat in AWS Sagemaker.
Error "gpt_bigcode" for lacking the last transformers version, that is not compatible with Sagemaker.

Any ideas to hack this?

Thanks

ququcana

May 25, 2023

Hey, same here)
there are some references that transformers version 4.28.1 is required to pass "gpt_bigcode" error.

As an idea:

there is a dlc with transformers 4.28.1 just released here, and the model could be created with help of sagemaker.huggingface (and boto3) python package(s) with any given image uri.

I've just tried, it gave me another error:
ValueError: Could not load model /.sagemaker/mms/models/bigcode__starcoder with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.gpt_bigcode.modeling_gpt_bigcode.GPTBigCodeForCausalLM'>).

zkrider

Jul 31, 2023

It worked by putting it on the AWS instance type: ml.g4dn.12xlarge and setting SM_NUM_GPUS: "4"

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment