DrDavis's picture
Upload folder using huggingface_hub
17c6d62 verified

Auto 클래슀[[auto-classes]]

λ§Žμ€ 경우, μ‚¬μš©ν•˜λ €λŠ” μ•„ν‚€ν…μ²˜λŠ” from_pretrained() λ©”μ†Œλ“œμ—μ„œ μ œκ³΅ν•˜λŠ” 사전 ν›ˆλ ¨λœ λͺ¨λΈμ˜ μ΄λ¦„μ΄λ‚˜ κ²½λ‘œλ‘œλΆ€ν„° μœ μΆ”ν•  수 μžˆμŠ΅λ‹ˆλ‹€. AutoClassesλŠ” 이 μž‘μ—…μ„ μœ„ν•΄ μ‘΄μž¬ν•˜λ©°, 사전 ν•™μŠ΅λœ λͺ¨λΈ κ°€μ€‘μΉ˜/ꡬ성/단어사전에 λŒ€ν•œ 이름/경둜λ₯Ό μ œκ³΅ν•˜λ©΄ μžλ™μœΌλ‘œ κ΄€λ ¨ λͺ¨λΈμ„ κ°€μ Έμ˜€λ„λ‘ λ„μ™€μ€λ‹ˆλ‹€.

[AutoConfig], [AutoModel], [AutoTokenizer] 쀑 ν•˜λ‚˜λ₯Ό μΈμŠ€ν„΄μŠ€ν™”ν•˜λ©΄ ν•΄λ‹Ή μ•„ν‚€ν…μ²˜μ˜ 클래슀λ₯Ό 직접 μƒμ„±ν•©λ‹ˆλ‹€. 예λ₯Ό λ“€μ–΄,

model = AutoModel.from_pretrained("google-bert/bert-base-cased")

μœ„ μ½”λ“œλŠ” [BertModel]의 μΈμŠ€ν„΄μŠ€μΈ λͺ¨λΈμ„ μƒμ„±ν•©λ‹ˆλ‹€.

각 μž‘μ—…μ— λŒ€ν•΄ ν•˜λ‚˜μ˜ AutoModel ν΄λž˜μŠ€κ°€ 있으며, 각각의 λ°±μ—”λ“œ(PyTorch, TensorFlow λ˜λŠ” Flax)에 ν•΄λ‹Ήν•˜λŠ” ν΄λž˜μŠ€κ°€ μ‘΄μž¬ν•©λ‹ˆλ‹€.

μžλ™ 클래슀 ν™•μž₯[[extending-the-auto-classes]]

각 μžλ™ ν΄λž˜μŠ€λŠ” μ‚¬μš©μžμ˜ μ»€μŠ€ν…€ 클래슀둜 ν™•μž₯될 수 μžˆλŠ” λ©”μ†Œλ“œλ₯Ό κ°€μ§€κ³  μžˆμŠ΅λ‹ˆλ‹€. 예λ₯Ό λ“€μ–΄, NewModelμ΄λΌλŠ” μ»€μŠ€ν…€ λͺ¨λΈ 클래슀λ₯Ό μ •μ˜ν–ˆλ‹€λ©΄, NewModelConfigλ₯Ό μ€€λΉ„ν•œ ν›„ λ‹€μŒκ³Ό 같이 μžλ™ ν΄λž˜μŠ€μ— μΆ”κ°€ν•  수 μžˆμŠ΅λ‹ˆλ‹€:

from transformers import AutoConfig, AutoModel

AutoConfig.register("new-model", NewModelConfig)
AutoModel.register(NewModelConfig, NewModel)

μ΄ν›„μ—λŠ” 일반적으둜 μžλ™ 클래슀λ₯Ό μ‚¬μš©ν•˜λŠ” κ²ƒμ²˜λŸΌ μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€!

λ§Œμ•½ NewModelConfigκ°€ [~transformers.PretrainedConfig]의 μ„œλΈŒν΄λž˜μŠ€λΌλ©΄, ν•΄λ‹Ή model_type 속성이 등둝할 λ•Œ μ‚¬μš©ν•˜λŠ” ν‚€(μ—¬κΈ°μ„œλŠ” "new-model")와 λ™μΌν•˜κ²Œ μ„€μ •λ˜μ–΄ μžˆλŠ”μ§€ ν™•μΈν•˜μ„Έμš”.

λ§ˆμ°¬κ°€μ§€λ‘œ, NewModel이 [PreTrainedModel]의 μ„œλΈŒν΄λž˜μŠ€λΌλ©΄, ν•΄λ‹Ή config_class 속성이 등둝할 λ•Œ μ‚¬μš©ν•˜λŠ” 클래슀(μ—¬κΈ°μ„œλŠ” NewModelConfig)와 λ™μΌν•˜κ²Œ μ„€μ •λ˜μ–΄ μžˆλŠ”μ§€ ν™•μΈν•˜μ„Έμš”.

AutoConfig[[transformers.AutoConfig]]

[[autodoc]] AutoConfig

AutoTokenizer[[transformers.AutoTokenizer]]

[[autodoc]] AutoTokenizer

AutoFeatureExtractor[[transformers.AutoFeatureExtractor]]

[[autodoc]] AutoFeatureExtractor

AutoImageProcessor[[transformers.AutoImageProcessor]]

[[autodoc]] AutoImageProcessor

AutoProcessor[[transformers.AutoProcessor]]

[[autodoc]] AutoProcessor

일반적인 λͺ¨λΈ 클래슀[[generic-model-classes]]

λ‹€μŒ μžλ™ ν΄λž˜μŠ€λ“€μ€ νŠΉμ • ν—€λ“œ 없이 κΈ°λ³Έ λͺ¨λΈ 클래슀λ₯Ό μΈμŠ€ν„΄μŠ€ν™”ν•˜λŠ” 데 μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

AutoModel[[transformers.AutoModel]]

[[autodoc]] AutoModel

TFAutoModel[[transformers.TFAutoModel]]

[[autodoc]] TFAutoModel

FlaxAutoModel[[transformers.FlaxAutoModel]]

[[autodoc]] FlaxAutoModel

일반적인 사전 ν•™μŠ΅ 클래슀[[generic-pretraining-classes]]

λ‹€μŒ μžλ™ ν΄λž˜μŠ€λ“€μ€ 사전 ν›ˆλ ¨ ν—€λ“œκ°€ ν¬ν•¨λœ λͺ¨λΈμ„ μΈμŠ€ν„΄μŠ€ν™”ν•˜λŠ” 데 μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

AutoModelForPreTraining[[transformers.AutoModelForPreTraining]]

[[autodoc]] AutoModelForPreTraining

TFAutoModelForPreTraining[[transformers.TFAutoModelForPreTraining]]

[[autodoc]] TFAutoModelForPreTraining

FlaxAutoModelForPreTraining[[transformers.FlaxAutoModelForPreTraining]]

[[autodoc]] FlaxAutoModelForPreTraining

μžμ—°μ–΄ 처리[[natural-language-processing]]

λ‹€μŒ μžλ™ ν΄λž˜μŠ€λ“€μ€ μ•„λž˜μ˜ μžμ—°μ–΄ 처리 μž‘μ—…μ— μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

AutoModelForCausalLM[[transformers.AutoModelForCausalLM]]

[[autodoc]] AutoModelForCausalLM

TFAutoModelForCausalLM[[transformers.TFAutoModelForCausalLM]]

[[autodoc]] TFAutoModelForCausalLM

FlaxAutoModelForCausalLM[[transformers.FlaxAutoModelForCausalLM]]

[[autodoc]] FlaxAutoModelForCausalLM

AutoModelForMaskedLM[[transformers.AutoModelForMaskedLM]]

[[autodoc]] AutoModelForMaskedLM

TFAutoModelForMaskedLM[[transformers.TFAutoModelForMaskedLM]]

[[autodoc]] TFAutoModelForMaskedLM

FlaxAutoModelForMaskedLM[[transformers.FlaxAutoModelForMaskedLM]]

[[autodoc]] FlaxAutoModelForMaskedLM

AutoModelForMaskGeneration[[transformers.AutoModelForMaskGeneration]]

[[autodoc]] AutoModelForMaskGeneration

TFAutoModelForMaskGeneration[[transformers.TFAutoModelForMaskGeneration]]

[[autodoc]] TFAutoModelForMaskGeneration

AutoModelForSeq2SeqLM[[transformers.AutoModelForSeq2SeqLM]]

[[autodoc]] AutoModelForSeq2SeqLM

TFAutoModelForSeq2SeqLM[[transformers.TFAutoModelForSeq2SeqLM]]

[[autodoc]] TFAutoModelForSeq2SeqLM

FlaxAutoModelForSeq2SeqLM[[transformers.FlaxAutoModelForSeq2SeqLM]]

[[autodoc]] FlaxAutoModelForSeq2SeqLM

AutoModelForSequenceClassification[[transformers.AutoModelForSequenceClassification]]

[[autodoc]] AutoModelForSequenceClassification

TFAutoModelForSequenceClassification[[transformers.TFAutoModelForSequenceClassification]]

[[autodoc]] TFAutoModelForSequenceClassification

FlaxAutoModelForSequenceClassification[[transformers.FlaxAutoModelForSequenceClassification]]

[[autodoc]] FlaxAutoModelForSequenceClassification

AutoModelForMultipleChoice[[transformers.AutoModelForMultipleChoice]]

[[autodoc]] AutoModelForMultipleChoice

TFAutoModelForMultipleChoice[[transformers.TFAutoModelForMultipleChoice]]

[[autodoc]] TFAutoModelForMultipleChoice

FlaxAutoModelForMultipleChoice[[transformers.FlaxAutoModelForMultipleChoice]]

[[autodoc]] FlaxAutoModelForMultipleChoice

AutoModelForNextSentencePrediction[[transformers.AutoModelForNextSentencePrediction]]

[[autodoc]] AutoModelForNextSentencePrediction

TFAutoModelForNextSentencePrediction[[transformers.TFAutoModelForNextSentencePrediction]]

[[autodoc]] TFAutoModelForNextSentencePrediction

FlaxAutoModelForNextSentencePrediction[[transformers.FlaxAutoModelForNextSentencePrediction]]

[[autodoc]] FlaxAutoModelForNextSentencePrediction

AutoModelForTokenClassification[[transformers.AutoModelForTokenClassification]]

[[autodoc]] AutoModelForTokenClassification

TFAutoModelForTokenClassification[[transformers.TFAutoModelForTokenClassification]]

[[autodoc]] TFAutoModelForTokenClassification

FlaxAutoModelForTokenClassification[[transformers.FlaxAutoModelForTokenClassification]]

[[autodoc]] FlaxAutoModelForTokenClassification

AutoModelForQuestionAnswering[[transformers.AutoModelForQuestionAnswering]]

[[autodoc]] AutoModelForQuestionAnswering

TFAutoModelForQuestionAnswering[[transformers.TFAutoModelForQuestionAnswering]]

[[autodoc]] TFAutoModelForQuestionAnswering

FlaxAutoModelForQuestionAnswering[[transformers.FlaxAutoModelForQuestionAnswering]]

[[autodoc]] FlaxAutoModelForQuestionAnswering

AutoModelForTextEncoding[[transformers.AutoModelForTextEncoding]]

[[autodoc]] AutoModelForTextEncoding

TFAutoModelForTextEncoding[[transformers.TFAutoModelForTextEncoding]]

[[autodoc]] TFAutoModelForTextEncoding

컴퓨터 λΉ„μ „[[computer-vision]]

λ‹€μŒ μžλ™ ν΄λž˜μŠ€λ“€μ€ μ•„λž˜μ˜ 컴퓨터 λΉ„μ „ μž‘μ—…μ— μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

AutoModelForDepthEstimation[[transformers.AutoModelForDepthEstimation]]

[[autodoc]] AutoModelForDepthEstimation

AutoModelForImageClassification[[transformers.AutoModelForImageClassification]]

[[autodoc]] AutoModelForImageClassification

TFAutoModelForImageClassification[[transformers.TFAutoModelForImageClassification]]

[[autodoc]] TFAutoModelForImageClassification

FlaxAutoModelForImageClassification[[transformers.FlaxAutoModelForImageClassification]]

[[autodoc]] FlaxAutoModelForImageClassification

AutoModelForVideoClassification[[transformers.AutoModelForVideoClassification]]

[[autodoc]] AutoModelForVideoClassification

AutoModelForKeypointDetection[[transformers.AutoModelForKeypointDetection]]

[[autodoc]] AutoModelForKeypointDetection

AutoModelForMaskedImageModeling[[transformers.AutoModelForMaskedImageModeling]]

[[autodoc]] AutoModelForMaskedImageModeling

TFAutoModelForMaskedImageModeling[[transformers.TFAutoModelForMaskedImageModeling]]

[[autodoc]] TFAutoModelForMaskedImageModeling

AutoModelForObjectDetection[[transformers.AutoModelForObjectDetection]]

[[autodoc]] AutoModelForObjectDetection

AutoModelForImageSegmentation[[transformers.AutoModelForImageSegmentation]]

[[autodoc]] AutoModelForImageSegmentation

AutoModelForImageToImage[[transformers.AutoModelForImageToImage]]

[[autodoc]] AutoModelForImageToImage

AutoModelForSemanticSegmentation[[transformers.AutoModelForSemanticSegmentation]]

[[autodoc]] AutoModelForSemanticSegmentation

TFAutoModelForSemanticSegmentation[[transformers.TFAutoModelForSemanticSegmentation]]

[[autodoc]] TFAutoModelForSemanticSegmentation

AutoModelForInstanceSegmentation[[transformers.AutoModelForInstanceSegmentation]]

[[autodoc]] AutoModelForInstanceSegmentation

AutoModelForUniversalSegmentation[[transformers.AutoModelForUniversalSegmentation]]

[[autodoc]] AutoModelForUniversalSegmentation

AutoModelForZeroShotImageClassification[[transformers.AutoModelForZeroShotImageClassification]]

[[autodoc]] AutoModelForZeroShotImageClassification

TFAutoModelForZeroShotImageClassification[[transformers.TFAutoModelForZeroShotImageClassification]]

[[autodoc]] TFAutoModelForZeroShotImageClassification

AutoModelForZeroShotObjectDetection[[transformers.AutoModelForZeroShotObjectDetection]]

[[autodoc]] AutoModelForZeroShotObjectDetection

μ˜€λ””μ˜€[[audio]]

λ‹€μŒ μžλ™ ν΄λž˜μŠ€λ“€μ€ μ•„λž˜μ˜ μ˜€λ””μ˜€ μž‘μ—…μ— μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

AutoModelForAudioClassification[[transformers.AutoModelForAudioClassification]]

[[autodoc]] AutoModelForAudioClassification

TFAutoModelForAudioClassification[[transformers.TFAutoModelForAudioClassification]]

[[autodoc]] TFAutoModelForAudioClassification

AutoModelForAudioFrameClassification[[transformers.AutoModelForAudioFrameClassification]]

[[autodoc]] AutoModelForAudioFrameClassification

AutoModelForCTC[[transformers.AutoModelForCTC]]

[[autodoc]] AutoModelForCTC

AutoModelForSpeechSeq2Seq[[transformers.AutoModelForSpeechSeq2Seq]]

[[autodoc]] AutoModelForSpeechSeq2Seq

TFAutoModelForSpeechSeq2Seq[[transformers.TFAutoModelForSpeechSeq2Seq]]

[[autodoc]] TFAutoModelForSpeechSeq2Seq

FlaxAutoModelForSpeechSeq2Seq[[transformers.FlaxAutoModelForSpeechSeq2Seq]]

[[autodoc]] FlaxAutoModelForSpeechSeq2Seq

AutoModelForAudioXVector[[transformers.AutoModelForAudioXVector]]

[[autodoc]] AutoModelForAudioXVector

AutoModelForTextToSpectrogram[[transformers.AutoModelForTextToSpectrogram]]

[[autodoc]] AutoModelForTextToSpectrogram

AutoModelForTextToWaveform[[transformers.AutoModelForTextToWaveform]]

[[autodoc]] AutoModelForTextToWaveform

λ©€ν‹°λͺ¨λ‹¬[[multimodal]]

λ‹€μŒ μžλ™ ν΄λž˜μŠ€λ“€μ€ μ•„λž˜μ˜ λ©€ν‹°λͺ¨λ‹¬ μž‘μ—…μ— μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

AutoModelForTableQuestionAnswering[[transformers.AutoModelForTableQuestionAnswering]]

[[autodoc]] AutoModelForTableQuestionAnswering

TFAutoModelForTableQuestionAnswering[[transformers.TFAutoModelForTableQuestionAnswering]]

[[autodoc]] TFAutoModelForTableQuestionAnswering

AutoModelForDocumentQuestionAnswering[[transformers.AutoModelForDocumentQuestionAnswering]]

[[autodoc]] AutoModelForDocumentQuestionAnswering

TFAutoModelForDocumentQuestionAnswering[[transformers.TFAutoModelForDocumentQuestionAnswering]]

[[autodoc]] TFAutoModelForDocumentQuestionAnswering

AutoModelForVisualQuestionAnswering[[transformers.AutoModelForVisualQuestionAnswering]]

[[autodoc]] AutoModelForVisualQuestionAnswering

AutoModelForVision2Seq[[transformers.AutoModelForVision2Seq]]

[[autodoc]] AutoModelForVision2Seq

TFAutoModelForVision2Seq[[transformers.TFAutoModelForVision2Seq]]

[[autodoc]] TFAutoModelForVision2Seq

FlaxAutoModelForVision2Seq[[transformers.FlaxAutoModelForVision2Seq]]

[[autodoc]] FlaxAutoModelForVision2Seq